SF's population density isn't Manhattan's, but it's quite high overall, one of the highest in the US. The city in total has a bit over 17,000 people per square mile; that's 3x the density of Seattle and 6x that of Austin, for example. The parts where demand is highest (the eastern half) are even higher, e.g. about 25,000/sq. mi in the Mission.
Compare that to, say, Palo Alto, where housing is in strong demand (due to Stanford and startups) and yet they've only allowed density to reach a paltry 2,500/sq.mi. If they allowed high-density housing near downtown Palo Alto and Stanford, it'd not only improve things on the peninsula, but also reduce some of the pressure on SF housing from people who currently reverse-commute.
It's difficult to compare population densities between cities because city boundaries don't mean the same thing. E.g. Brooklyn is part of New York, while Oakland isn't part of San Francisco, but Brooklyn and Oakland have similar geographic functions.
As of 2000, about 871,000 people (one San Francisco) lived in Chicago neighborhoods cumulatively as dense as the Mission (~25k/square mile). About 2.4 million people (three San Franciscos) lived in neighborhoods cumulatively as dense as San Francisco (~17K/square mile). Put another way, if San Francisco (~47 square miles) was as dense as the densest 47 square miles of Chicago, it would have 23,500 people per square mile and 1.1 million people.
Chicago is officially much less dense than San Francisco because the way the boundaries are defined, the city proper stretches 20 miles along the lake shore, two-thirds of the way to Gary, Indiana. San Francisco, meanwhile, ends quite abruptly part-way down the Peninsula. But if you look at the population density weighted by the populations of neighborhoods, Chicago is more dense.
Interestingly, Chicago has no reason to be so dense. Aside from the lake, it's flat, easily-built land as far as the eye can see. It's just that historically Chicago has had lax zoning regulation and that has resulted in more density.
You could certainly do that. By that measure San Francisco is actually denser than New York at 6,266 per square mile versus 5,319 per square mile (and 3,524 for Chicago). L.A. wins it all at 6,999 per square mile. Which really puts a hole in the arguments that the Bay Area and L.A. are not dense enough for commuter rail.
It depends on what you're trying to measure, really. If you're talking about space constraints driving up housing prices in the core city, I think it's useful to look at density in the core city and whether it could be higher to create more supply.
Surely these utterly fall apart (perhaps that is what you meant to begin with) when considering actual usage rather than arbitrary geographical boundaries.
Oakland is very much not the geographical equivalent of Brooklyn (you might be able to make some sort of argument for the Bronx or Staten Island). San Francisco is not more densely populated than Manhattan, neither is, by a long stretch LA.
> SF's population density isn't Manhattan's, but it's quite high overall
San Francisco has half the population density of Brooklyn and less than Queens or the Bronx. It's also less than Somerville, Massachuetts or West Hollywood in LA county.
People talk about having more people in San Francisco and they think of Manhattan, but there's actually a lot of room for increased density without looking like Manhattan.
That's true, Manhattan-style skycrapers aren't the only option. The transit-reachable parts of SF are already denser than some of what you mention as comparable, though: the Mission has 50% higher population density than West Hollywood.
One problem is that density is much lower in places like the Outer Sunset, in part because there's no cross-town subway. Total SF population could be increased significantly if the western half's population density could be brought up to what's already the case in the eastern half.
It's an easy problem to solve: allow more housing in the Outer Sunset. The demand for housing in SF is fierce, and throwing up some towers (or hell, just a-few-stories-tall buildings) will create a huge population boom.
Which will beget the subway.
As it is there is neither the political will, nor the practical population, to support such a thing.
Compare that to, say, Palo Alto, where housing is in strong demand (due to Stanford and startups) and yet they've only allowed density to reach a paltry 2,500/sq.mi. If they allowed high-density housing near downtown Palo Alto and Stanford, it'd not only improve things on the peninsula, but also reduce some of the pressure on SF housing from people who currently reverse-commute.