I've been "harassed" a few times over seas. Usually it involves accidently photographing near police stations or military installations, which is a pretty obvious no-no, except that the buildings were pretty non-descript and I had no idea what they were at the time.

It's only happened a few times: someone comes out and either tells me to stop, or yells at me in a language I don't speak very well, and I say "sorry" in whatever language and continue walking around.

I find that the trouble happens when photographers/travelers try to argue with the people harassing them.

Usually most the people I encounter just make a joke, smile, and encourage me to take photos.

Funny how far a smile gets you when you're traveling.