I disagree that is has little effect. The way we communicate is a central and essential aspect of us as a species. Even the little nods and the dog whistles matter. A dominate culture policing it's norm of communication onto another because the dominate one believes due to it being the prominent one it is thus superior leads to damaging and loss of cultural diversity and communication and perpetuates societal disunity and segregation.

Viewing swearing as "dumb" and thus people who swear are "dumb" is a really easy way to openly okay associating a whole group of people that may swear often as "dumb" or to selectively apply that standard to them.

You yourself are perfectly free to choose whatever words you would like to use; I am perfectly free to find language policing and censoring repugnant.