This is not true. We did make it an investment asset around the advent of agriculture. And there is a way to avoid it without freezing the market, which is called the Land Value Tax. As far as I know, no serious economist (from any ideological perspective) seems to actually dispute the functional fitness of this tax.

https://en.wikipedia.org/wiki/Land_value_tax?wprov=sfti1