If you already own a home. Otherwise housing is just more expensive. Lower interest rates mean houses can be sold at higher prices for exactly the reason you gave.
It comes down to what people that want to live in a particular area can afford to pay monthly for housing and taxes. As interest rates go down, bidding will increase on homes as people consider the monthly cost which is generally mortgage + interest + taxes.
The only real effect it has on first time buyers is coming up with a percentage based down payment. But 20% isn't a real requirement nowadays, but you will incur a PMI payment until you get to 20% equity.
But housing isn't more expensive, unless you want to pay cash for the home. Then yes, low interest rates aren't nice. But when interest rates are low, what's the point in paying cash? Someone offering you money below what you could make on that money in the open market is something you should take. Put enough down to give you a comfortable monthly payment (or get to 20% to avoid PMI) and diversify the rest of your money.