OK, set aside the judgement on rudeness for now. Twitter limits the number of people you can follow with daily limits and rate limits and a hard stop at 2,000. In order to follow more than 2,000 you need first to be followed by around 1,900 people. There are many people who would like to generate a large following. After all, the most influential/interesting people (and celebrities) are followed by millions. Therefore many people unfollow non-followers so they can move on to follow someone who does follow back. Think of it as a karma system that has real world relevance because the number of followers you have is public and often your Twitter handle is associated with your real identity. So, if you do not follow everyone who follows you will lose folowers. Unless you are Justin Bieber of course.

The rude part is that someone took the microsecond required to follow you and now they have to use a tool to find their unfollowers (you), unfollow you, and follow someone else.

Who you follow is a reading list for people who share your interests. Polluting it with people whom you follow just as a hat tip is rude. Also, following uninteresting people makes your stream unusable. In my book, keeping a high signal to noise ratio is the epitome of politeness. I guess that we don't use Twitter for the same reasons...