I really dislike posts in this vein, because they use statements like
"This post will teach you how to replace estimations and guesses with a Monte Carlo simulation."
as though the output of a Monte Carlo simulation were not an estimate, but some higher truth. The process of simulating many different outcomes with a touch of randomness gives the layperson the impression that we are really _doing_ something, when in fact we could have obtained the exact result (no confidence intervals needed) of Monte Carlo by quadrature (although admittedly that could be tedious).
In the end, the method outlined in the link is an estimate, like any other, and is not necessarily better or worse than any other estimate. It erects a straw person and then misuses mathematical terminology throughout.
I sort of agree with you although for different reasons. The problem with the MC approach IMO is not really that it is also an estimate, estimates are fine. However doing a Monte Carlo simulation is only reasonable if we have a good model. So now we have moved from we can't estimate (intuitively I guess) to how to find a good model, that is not trivial and the time spend on finding and verifying the model might take more time than what you are actually trying to do (just an (gu-)estimate on my part)
A Monte Carlo simulation is infinitely better than all the conventional "to estimate, take your first estimate and multiply it with 10x". And even better, your manager can do that without your help and this learn better how much you tend to over/under estimate projects without trying to force you to change your estimates to fit what he think it should be.
It also ensures that the final estimate becomes on average correct, which is much better than most projects which tends to under estimate the effort it will take. Although, under estimating projects is often a feature managers wants since it makes it easier to sell it to customers, ie "My engineers estimated we can remake Twitter in a weekend, I think we should focus on that!".
Anyway, the point is that if your manager complains about your estimations being on average off then he is just incompetent and should have applied a Monte Carlo simulation on it instead of asking you to perform an easily automatable task.
Indeed, "estimations" might not be the best word as you're still, in some way, estimating with the Monte Carlo approach. I thought of this differentiation because I consider the MC approach to yield a forecast, while an "estimation" is the typical term used in agile settings for "guessing how long/much work it will take.". I'll think of ways of making this distinction clearer.
As for the misused mathematical terminology I guess you're referring to "confidence intervals"? If so, I understand it also has a clear definition within statistics, but I used it in the broader sense. I'd be grateful if you could point out which terms you'd recommend replacing and what you think I could replace it for.
> Indeed, "estimations" might not be the best word as you're still, in some way, estimating with the Monte Carlo approach.
Not in some way -- literally. The samples are averaged, and as the number of samples goes to infinity, the average converges to some true value. In the case of one of the histograms you plot, you are averaging an indicator function .
> I thought of this differentiation because I consider the MC approach to yield a forecast, while an "estimation" is the typical term used in agile settings for "guessing how long/much work it will take.". I'll think of ways of making this distinction clearer.
It seems to me the distinction you care for is to provide not a single value ("estimate"), but a distribution of values. That's not specific to Monte Carlo.
> As for the misused mathematical terminology I guess you're referring to "confidence intervals"
That's certainly one of them. I even misunderstood what you were doing and said you don't need confidence intervals, but in the way you're using them those again can be obtained by quadrature.
My main gripe remains the language, and the presentation of the method. You could most likely obtain your results by hand, i.e. with pen and paper, and you would still get a distribution at the end of the day. The advantage of running Monte Carlo is that it simplifies this process. You could illustrate this by getting the distribution of the sum of two independent uniform random variables in two different ways: by doing a convolution, and by MC. Wow, MC was so much easier, anyone can do it, and it can handle arbitrarily complicated distributions (in principle).
I use Monte Carlo simulations because I am not that great at doing complex statistics.
It sounds fancier to say modeled using a Monte Carlo simulation, but in reality it is much easier than working with joint and conditional probabilities.
"This post will teach you how to replace estimations and guesses with a Monte Carlo simulation."
as though the output of a Monte Carlo simulation were not an estimate, but some higher truth. The process of simulating many different outcomes with a touch of randomness gives the layperson the impression that we are really _doing_ something, when in fact we could have obtained the exact result (no confidence intervals needed) of Monte Carlo by quadrature (although admittedly that could be tedious).
In the end, the method outlined in the link is an estimate, like any other, and is not necessarily better or worse than any other estimate. It erects a straw person and then misuses mathematical terminology throughout.