I'd rather just live next to well off people to be honest.

We do it on a national scale in most countries (visa policy). That's far harsher.