I've lived here off and on since 1993. I work for startups but haven't hit the lottery.
There's a lot of San Francisco (and Bay Area) out there that has nothing to do with the startup scene, and many, many people who have no connection to and couldn't care less about it.
I agree, SF area has a lot of non-tech-related people and activities that should be enough to at least enjoy your non-work hours, but same with other areas. Moved to San Diego, and now I'm happier all around.
Running joke between me and the rest of the startup team in SF was that every time we go to an event, people lead conversations by asking what stack I use, which version of Python, and why the heck I'm messing with XMPP. I understood the importance of seeking out non-techies.
I don't know about you, but people don't charge me money to talk to them. All I have to do to meet people outside of my work peer group is go somewhere different.
100% believe this.