>I don't see how a method that does not require independent replications can be considered science.
There is an implicit benefit-of-doubt here - that the scientists themselves have independently replicated the experiment before publishing the result. Most papers will list how many times an experiment was repeated by providing the sample size, number of biological replicates, number of technical replicates, orthogonal evidence etc.
>"the scientists themselves have independently replicated the experiment"
Is this independent though? One person/group repeating the same experiment is much less convincing evidence of a stable phenomenon and control of the experimental situation. I take independent replication to mean others getting similar results.
Going further, the ideal situation is when there is healthy rivalry (such as between universities), so the multiple research groups have incentive to find flaws with the claims of the others.
The dynamic there is that there's little incentive to replicate results - no journal is going to publish an article based on that. Why spend your time and money on that then?
That's a problem for those of us who think independent replication is a crucial part of the scientific method.
If no one is going to attempt replicating my research and there are substantial obstacles placed in the way of me replicating other work... why waste my time? It doesn't pay very well, and it seems I am forced into a position of contributing to pseudoscience.
Eventually only people who do not consider independent replication crucial will remain.
Independent replication does occur when someone furthers the research, usually as a preliminary test of current knowledge before diving into new experiments. I personally took great joy when what I had reported was used and confirmed in the preliminary experiments done by other labs for their further studies.
But there is a deeper philosophical issue with your idea of the scientific method :
Consider what is meant by independent replication. Does it mean that the experiment must be repeated from scratch with different material etc. such that it is an independent test from the previous attempt?
Or does it mean that it has to be repeated by different people as well ?
It raises a conundrum. If people are a factor that must be controlled for by this definition, no replication can ever occur since the exact individuals that performed the experiment will have changed by definition during this 'replication' attempt. The skill and knowledge of people is a factor that is notoriously difficult to objectively quantify. This often results in statements such as "X method/assay/procedure does not work well in our hands ".
On the other hand , if people are not a factor, then replication of the experiment by the same group of people will also qualify as an independent replicate.
>"Or does it mean that it has to be repeated by different people as well ?"
Yes. The purpose is to 1) demonstrate the experimental conditions have been mastered to the point they can be communicated effectively, and 2) demonstrate the phenomenon is stable in the face of any unknown factors specific to a place and time.
1. Effective communication of a protocol has no relevance to its efficacy. So that's a plainly unscientific test of the 'truthiness' of anything.
2. No phenomenon ever, scientific or not, can be guaranteed to be stable in the face of all 'unknown' factors. They are unknown, you cannot make any statement about that.
I understand where you are coming from, but that puts you right in the middle of the philosophical quagmire mentioned in my earlier comment. If you mean 'variation in people performing the experiment' by those 'unknown factors' then essentially you are making the assumption that the people are not a factor that is expected to influence the result. As such, even the same team repeating the experiment would suffice. Anyway, you get the point.
It's a time tested heuristic. If a result cannot be communicated well enough for others to replicate or if it strongly depends upon local conditions (doesn't really matter which reason), we should either focus on something else or figure out why. If there is no independent replication, there is no chance to learn either of the above and no reason to have confidence we know what is going on.
I see no quagmire, it is all very straightforward. I will not believe my own results until others replicate them. Even once is not enough to hang my hat on. What alternative approach do you use to judge whether an observation is worth theorizing about?
i don't disagree that independent replication is useful or crucial. the obstacle is one that is intrinsic to academia - 'publish or perish.'
it is logical for a researcher to devote their time and energy to research that will be published, and journals just don't devote space to independent verification efforts.
There is an implicit benefit-of-doubt here - that the scientists themselves have independently replicated the experiment before publishing the result. Most papers will list how many times an experiment was repeated by providing the sample size, number of biological replicates, number of technical replicates, orthogonal evidence etc.