Pretty much.

If you don't eject the extremist elements from your community, your community will be represented by them to outside world.

I do wonder what would happen with social media that had only the up/downvote system as soft moderation but without actual moderators smacking people around