I don't think this is a very good argument. The term blacks was likely arbitrarily chosen and wasn't meant in a derogatory or disrespectful manner.

Also, where in the world have you lived? I just moved out of Harlem a few months ago and experienced everything the OP mentioned and more. The fact is, Harlem (at least East Harlem) is a dangerous and predominately black community. Crime there is driven more by circumstance (upbringing, low income, environment, etc.) than skin color, obviously, but I don't blame anyone who wants to move out of a dangerous area.

Also, "don't live around poor people" is basically saying "why don't you just make more money?", and is incredibly thoughtless. I didn't choose to live in a crumbling walkup in the bad part of town, I had to because that was my only option, and I got out as soon as I could.