The justice system's mechanism for protecting from the "someone's speech cost you clients" scenario is libel (or slander). Both protect a person from defamatory, factual, untrue statements.
"That guy is an asshole" isn't one of those, and the only protection American law offers in that case is "People should think for themselves."
And, of course, the mirror image of the terribly stupid argument that if there isn't a criminal trial, no assault could have occurred is the argument that if the accused doesn't win a suit for libel, the accusation must be valid.
"That guy is an asshole" isn't one of those, and the only protection American law offers in that case is "People should think for themselves."