If you describe anyone with a college education as a nerd then the word loses almost all meaning, as it'd describe a huge number of people.

Well, the word nerd (and geek) used to mean someone who you'd expect to be better in their domain than your average college degree holder, so me having to defend nerds only shows we've already reversed the meaning.

Just as a counter point, I've never thought of nerd/geek as terms describing prowess in a field. I see 'geek' as a term to describe someone who is passionate about something that it isn't cool to be passionate about, or in a way that isn't concerned with being cool. Nerd would be basically the same, except it's taken on a stronger negative connotation, basically because it's used to imply lack of social skills, and perhaps more obsessive behaviour.