I think asking for location is perfectly sensible and the right thing to do. Track how many people say no, but if the opt out rate is too high, work on the wording to increase the rate before you allow manually entering a city.