Is it really the best job when the out of control expenses in those cities leave you no further ahead than someone working in the 'not the best job' elsewhere? Unless by best job you mean some kind of non-monetary metric, but I'm not sure that holds either. The best jobs for non-monteary reasons in my opinion are also found outside of those cities, but are definitely low paying due to the fact that everyone wants to do that kind of work.
It depends. You are right that you can probably be monetarily better off living in some smaller cheaper city and working as a developer for a non exciting company. Real estate prices in that city are probably something like 3-5 times lower than SF/London.
The downside is that selection of jobs is very lacking in places like that. If you are not happy with your job, or want to try something else (maybe go to a different field within IT), it might be very difficult to find anything new. Also in economic downturn like 2008 there will still be enough new jobs in big tech hubs but anything outside might dry up for couple of years.
Another factor is that even though everything is super expensive and you might be relatively poorer compared to somebody with 60% of your salary but living in a place with 40% cost of living, once you manage to get on a property ladder in a city like SF, given the constantly rising real estate prices, your house will be worth a lot once you repay your mortgage and can be a great way to retire later.
