Yes. See http://en.wikipedia.org/wiki/Standard_deviation . In that first graphical plot, the little σ after the number is lowercase Greek sigma. (The upper case sigma, which may be more recognizable, is Σ, which you may recognize as being used for summation.) Per the chart about halfway down the page, a 5 sigma result means that there is a 1 / 1,744,278 probability that it has come from pure chance.
Particle physicists generate data by the petabyte. If they used 3 sigma confidence they'd confidently declare all sorts of false things. We know this, because if you pay attention to particle physics you hear about all kinds of 3 sigmas that turn out to be spurious. (About every six months to a year or so, I'd say.) 5 is just where they start to feel comfortable, they'll take more if they can get it.
Your answer is good and sufficient. For people who aren't yet getting an intuitive feel: if 99% confidence means you're right 99% of the time (it doesn't quite mean that but lets go with it), then you'll still be wrong 1% of the time. If 3000 studies are published each year with that cutoff for significance, you should expect about 30 spurious results accepted as true each year. Ouch.
In human genetics, studies will meta-analyze each of the ~million SNPs tested across each study, rather than just picking the "candidate" SNPs that came to attention because of their association signal in any particular study. So the file drawer problem is still relevant, but the risk is greatly mitigated.
Because 3 sigma doesn't mean 99% chance they've found it; it means that, if it doesn't exist, there's a 1% chance of obtaining the results they did. Common statistical misconception.
Consider a situation where they test 1000 hypotheses, only 100 of which are actually true. Of the 900 false hypotheses, 9 will, by chance, achieve a 3 sigma "99% chance of being right" result. And hence something like 8.3% of the hypotheses tested as "true" will be false positives -- not 1%.
Particle physicists usually require 5 sigma for a discovery. They've apparently had plenty of 3 sigma results that didn't hold up, so they have high standards. 3 sigma to them now is merely the point where people start to get interested in your result.
Biology uses .95 because it's virtually impossible to get better results than that. But it also means that 1 out of every 20 biology papers are completely wrong. And then you have papers based on those....
Physics wants a much greater degree of certainty, and since they have the ability to get it they insist on it.
You have to make sure to have results that are far far better than the number of experiments you are running. Otherwise you are virtually guaranteed to find some result that will seem right, but isn't.
(For example a 1 in a million occurrence (per person per day) [would] happen about 7000 times every day.)
That "completely wrong" is a bit harsh on biologists.
Firstly, it is a lot harder to repeat experiments in biology than in particle physics, as (as far as we know), all electrons are the same.
Secondly, biologists will, in general, not make bold claims. A paper "a possible link between X and Z" that works with p>0.95 and states that further research is needed is not a lie; the popular press makes it a lie by changing it to "OMG: X CAUSES Y".
I know it's a lot harder to repeat experiments in biology than in particle physics - I said that in my post.
Just because someone uses weasel words ("possible") doesn't change the end result: It's a wrong result.
I'm not blaming them - I understand better results are not possible. But it doesn't change the fact that a tremendous number of results are wrong.
It doesn't help that they often search for very subtle results. "It helps, but only a little." It also doesn't help that everyone responds differently to things. It makes the research very hard.
Anyway, I was just explaining why p95 is not accepted anywhere else except biology - biology just doesn't have any other choice. They don't prefer such low results.
Different fields have different acceptable levels of significance. Social science and engineering fields generally except significance at 95% or 99% levels , physics has higher standards.
