Per Wikipedia (haven't had much time to research them otherwise), they're definitely violent (if not deadly), and using violence to stymie opinions you don't agree with is itself one of the defining traits of fascism.
Still, I can't quite get behind calling them terrorists. From what I've read about the Berkeley protests and Charlottesville, Antifa seems less concerned with pushing an agenda, and more concerned with opposing far-right supremacism, which itself is steeped in deadly intolerance. There's nothing wrong with fighting that kind of intolerance with intolerance.
No need to single-out Fascism. It's part of any violent extremist group.