I guess we're talking about software development. Because it depends a lot on the kind of the job (or maybe I'm just ignorant and someone else's job seems simpler ;) ).
But if we talk about software only, then estimating is hard, creating a strong and stable enough specification (that can be used for estimation) is hard, you have to sell it to the client - most clients are not willing to pay for it, for they don't understand that it would be needed in order to come up with a fixed cost. Even if they do understand the need and acknowledge the effort, they might say that if you create the spec and they don't like the final quote then the spec may not be valuable for them, because the next freelancer/company may not find it useful. (And it has some merit, though there are a lot of freelancers and companies who are willing to work and quote based on an existing detailed specification.)
And if you do the estimation (and create the quote) without quoting for the specification work then you'll put in a lot of uncompensated hours which you'll have to make up for with the paying clients, so you'll be more expensive. Especially if you find a client who shops around for quotes, plays the "lets do a spec" game (for which they don't pay), and then go with the specification (or even just the improved understanding) to the next few companies where they can provide a detailed spec so those companies can quote lower.
Oh, well and the client will compare the price tags (obviously and rightly so), so they will more likely to be choosing the offer that has the larger estimation error (as long as it's an underestimation, of course). And you can say that competition takes care of this (whoever is constantly underestimating, that constant underestimation will actually be their real price), but the above points still hold. It's just not worth it. Definitely not worth it if you don't have the client paying for the specification/planning that's needed for the quote.
On the other side, if you ask a lot of good questions and give a quote that's within budget, the client is going to be suspicious of another quote where similar insightful questions weren't asked first.
If another quote is very cheap, the client will probably question if that's going to reflect the quality of the work.
There's also risk that the developer they take the requirements + design doesn't really understand what they're getting into and can't execute.
I don't think there's any easy answers, but generally you want to work on attracting clients that are more interested in high quality work over the cheapest price where shopping around like this isn't a good use of their time. Chatting about rough budgets first and making sure the client is going to profit from the work helps too.
It's great when you get to charge for the planning phase though because you're not feeling rushed into committing to a quote with too many unknowns that might not really solve their problem. If you explain why it's risky to quote with so little information, some clients will listen and aren't going to go with someone asking nothing and offering a cheap price.
Yes, good questions are important in order to convince a client (whether the project is hourly or fixed price).
> There's also risk that the developer they take the requirements + design doesn't really understand what they're getting into and can't execute.
Yes, and I've mentioned that. But what matters is if the client is aware of it or not.
Yes, you always want to aim for quality clients, still with all this planning and estimation, even with the best intentions I think that a lot of time gets wasted on lost bids.
> It's great when you get to charge for the planning phase though
Now that I think back, I had exactly one project like this, more than a decade ago. I've created the requirement specification based on client interviews, they paid for it then sent it to a few competing companies and all of us made a proposal. We didn't win it in the end, but at least that was a pretty fair and transparent process. (The company was owned by Deutsche Telekom, so it wasn't exactly an accident.)
But if we talk about software only, then estimating is hard, creating a strong and stable enough specification (that can be used for estimation) is hard, you have to sell it to the client - most clients are not willing to pay for it, for they don't understand that it would be needed in order to come up with a fixed cost. Even if they do understand the need and acknowledge the effort, they might say that if you create the spec and they don't like the final quote then the spec may not be valuable for them, because the next freelancer/company may not find it useful. (And it has some merit, though there are a lot of freelancers and companies who are willing to work and quote based on an existing detailed specification.)
And if you do the estimation (and create the quote) without quoting for the specification work then you'll put in a lot of uncompensated hours which you'll have to make up for with the paying clients, so you'll be more expensive. Especially if you find a client who shops around for quotes, plays the "lets do a spec" game (for which they don't pay), and then go with the specification (or even just the improved understanding) to the next few companies where they can provide a detailed spec so those companies can quote lower.
Oh, well and the client will compare the price tags (obviously and rightly so), so they will more likely to be choosing the offer that has the larger estimation error (as long as it's an underestimation, of course). And you can say that competition takes care of this (whoever is constantly underestimating, that constant underestimation will actually be their real price), but the above points still hold. It's just not worth it. Definitely not worth it if you don't have the client paying for the specification/planning that's needed for the quote.