The other side of that coin (which is far more prevalent historically) is that speech restrictions are used to stifle dissent and repress minority groups. I.e. illegalizing anti-war or minority rights speech.
You make the mistake of assuming that those in power will only be preventing speech you agree with rather than muzzling you.
Honestly, I would prefer that over the ambiguity. I'd much rather live in a state in America where being a Nazi is illegal with the side effect of knowing, unequivocally, that I need to leave another state because my existence is in peril.
You make the mistake of assuming that those in power will only be preventing speech you agree with rather than muzzling you.