It's because the higher you live, the greater you will see the beauty of the world and so, the less you are already self realized, the more you will feel a bigger discrepancy with it. And it can hurt.