Thomas Kuhn, the famous philosopher and historian of science (and physicist by training), wrote all about this back in 1962 when he published The Structure of Scientific Revolutions. This book brought us the term "paradigm shift", which he argued typically only happened when older scientists retired, leaving younger scientists to pursue avenues of research that disagreed with the entrenched orthodoxy, without the threat of being denied tenure.
I.e., science as never been the direct route from hypothesis to proven theory that they teach us in high school.
Remember when relativity didn't need a generation of physicists to die before it was accepted? How about quantum mechanics?
The big problem I have with Kuhn's idea of paradigm-shift-by-mass-retirement is that there are a bunch of examples in the past century of really big upsets in a number of fields that were accepted without dramatic wailing and gnashing of teeth. They just don't get mentioned so much when the idea of paradigm shift comes up -- classic confirmation bias.
(If you want to see big changes in action today, look at molecular biology. Now there is a lively field! It's kind of disconcerting how often they have to update their textbooks, especially at higher levels.)
I'm not sure that it would be correct to interpret Kuhn as stating that paradigm shifts only occur upon death. His point was rather than science often doesn't proceed in the orderly way that we've typically been led to believe. Often does not preclude sometimes, or even frequently.
I don't know much about molecular biology (other than currently working on RNA Interference now for a living), but sometimes, I imagine, that there is just too much solid data for a shift, that the orthodoxists don't need to die to see the light. Or maybe these putative shifts don't really involve shifting your entire mindset, which was required for things like the shift from classical mechanics to quantum mechanics, or the shift from behavioral psychology to cognitive psychology. Kuhn's model of paradigm shift only applies if the paradigm has to change--not if there's just a lot of churning of the facts.
The shift from behaviorism to cognitivism was one that occurred relatively recently in the history of science, and it certainly involved dramatic wailing and the gnashing of teeth. In fact, there are still behaviorist holdouts.
Re relativity and quantum mechanics, these would be two of Kuhn's prime examples, so I'm not sure what you are saying with respect to them. Einstein didn't win a Nobel Prize for Relativity, because it still wasn't accepted well enough even all those decades later, when he did win his Nobel prize.
A current dramatic shift in biology (with wailing and the teeth and everything) is the fight between the people who favor group selection vs. the people who think group selection is bollocks and kin selection is to be preferred.
This has been going on for the last 50 years, with kin selectionists "winning", but E. O. Wilson recently published a book defending group-selection which Richard Dawkins ripped apart, so right now the fight has come back.
I think we don't see any major paradigm shifts in molecular biology (or my field, bioinformatics) because these aren't "declarative" per se, but rather "explorative" - meaning that there are no big theoretical frameworks like evolutionary biology has, but rather tiny steps of fiddling around in the dark and _then_ using theory to explain what was found.
That's actually the example Kuhn uses! And Kuhn didn't originate the idea in that case. He quotes Max Planck, one of the originators of quantum physics, who said something similar in his autobiography, about how quantum physics only came to be accepted after the old guard died out: A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.
Maybe I'm misreading your first sentence, but both relativity and quantum mechanics did require a generation of physicists to retire before their acceptance reached consensus. Even some of the proponents of quantum mechanics questioned its validity.
I had actually written "died off" rather than "retired", as that's how I recalled it being stated when I read The Structure of Scientific Revolutions. But then I couldn't recall a good reason why they had to die, and not just retire, so I changed the wording. But maybe even retired tenured scientists exert too much influence.
I love the Max Planck quote, though. I hadn't heard it before.
My bad theory is that Thomas Kuhn's theory of "paradigm shift" itself could be an example of bad theory as presented in the original article. As in, there might not be too tough to find a few obvious counter examples, essentially (proving?) the theory wrong...
The effect it calls into question was promoted by several articles that made it to the front page around here recently. For instance, http://news.ycombinator.com/item?id=4121859
"There's a myth out there that has gained the status of a cliché: that scientists love proving themselves wrong, that the first thing they do after constructing a hypothesis is to try to falsify it."
This sounds horribly wrong to me. Isn't it the "myth" that the scientific community is self-correcting, not individuals?
And not even that is necessarily true. I'm a physicist, and I would be wary of trying to prove any commonly accepted theory wrong - even if all my data points that way. Contrary to popular belief, getting data that is too non-mainstream and trying to defend it is often academic suicide in science. While science is one of the most rational fields to be working in, this is still a sad but true fact about the human condition.
Luckily, the fact that individual scientists are subject to the confirmation bias does not greatly affect the workings of science as a whole... for as soon as an interesting theory is developed by some scientist, hordes of other scientists will try to prove it false. For whatever reason...
This just is not true to the degree that people are typically led to believe. As I mentioned elsewhere on this page, Thomas Kuhn completely demolished this idealized version of how science works when he published The Structure of Scientific Revolutions in 1962.
The particular field that I know best where Kuhn's critique applies is that of psychology. For many decades, Skinnerian behavioral psychology was the orthodoxy for the scientific approach to psychology. Back in the day, if a scientist wanted to pursue a more cognitive approach, he would be squeezed out of the field. The rationale for this from the behaviorists was that cognitivism was anti-scientific because it posited complex internal mental states that could not be directly observed, which was considered to be counter to Empiricism, and hence anti-scientific. Behavioral psychology was based on the premise that mental states are very simple, and therefore that nearly everything important can be easily measured by directly observing behavior.
In their enthusiasm to be "scientific" and promote objective measurement, the behaviorists missed something very important: That their model was completely wrong. Wanting something to be easy to measure doesn't make it so. And the cognitivist claim that people have complex internal states is not anti-scientific, despite the protests of the behaviorists.
The "paradigm shift" did eventually come, when no other than Noam Chomsky published a devastating criticism of behaviorism that opened the door for the cognitive approach to be taken seriously. But this didn't happen without a fight that had lasted decades, where many careers were needlessly ruined, and the entire field was held back by decades.
The fact that Kuhn was so right on is seen by the fact that Kuhn's book was actually written before the Behavoirsim -> Cognitive Psychology shakeup was complete. The most prominent example in Kuhn's book, I believe, is for the rise of quantum mechanics.
I'm puzzled by this. As far as I can understand from your argument, a bunch of scientists didn't apply scientific method as well as they could. Meanwhile, another bunch did apply scientific method more objectively. And objectivity won out.
Where is the problem? There is an implicit, and idealistic, assumption here that science is flawed because the implementation scientists have is less than perfect and new ideas are not accepted instantly - despite there being very good reasons for this, and, in the long run, all of this working perfectly well to advance human knowledge.
I don't think that Kuhn described this process as "flawed". He was an historian wo revealed for us a pattern that often plays itself out in actuality, where for scientific progress to continue, the proponents of the current orthodoxy have to grow old and retire, in order to allow the new paradigm in. This process is completely different from what we are typically taught.
Also, what was not so apparent pre-Kuhn is that proponents of the new paradigm were often forced out of the field altogether if the time was not yet right.
To conclude that this means that science is "flawed" is a value judgement. But it certainly means that it is not an idealized puck on a frictionless plain.
This is actually very true. During my graduate work in chemistry, I learned how to present well because anytime you present something to a group of scientists, they immediately question every single assumption or scientific result that you obtained.
Where are the hordes of other scientists trying to prove Darwinism false?
I hope they are reading the prior literature on the science of evolution, for example as assembled in the "29+ Evidences for Macroevolution: The Scientific Case for Common Descent,"
so that they understand what "Darwinism" is today after 150 years of scientists checking facts and refining the understanding of the origin of species. Alas, there are still spectacular examples of people who purport to be scientists who disagree with Darwin
I hope they are reading the critiques, but as your comment proves, scientists trying to prove Darwinism false are just people purporting to be scientists.
Did you know that Darwin wrote 6 additional editions of Origin of Species [1] to address his critics? That there are many possible ways to disprove evolution by natural selection [2]?
I think those hordes assembled in the 1860's, I do not know what the half-life of a disproving scientist horde is but I suspect they have dispersed in disappointment quite some time ago.
In the same place as the hordes of other scientists trying to prove Einsteinism false: in the history books.
(No, real scientists don't call it Einsteinism, they call it relativity. Real scientists tend to say "evolution" or "natural selection" rather than Darwinism, too.)
in the United Kingdom the term has no negative connotations, being freely used as a short hand for the body of theory dealing with evolution, and in particular, evolution by natural selectionhttp://en.wikipedia.org/wiki/Darwinism
It's important to get a perspective on "theory" and "model" in science, if only because it's pivotal to understanding Kuhn's work and how he reconciled it with the more-famous account due to Popper that science works by falsification rather than confirmation.
The point is that a good theory, like quantum mechanics or Newtonian mechanics or atomic theory or heliocentrism, is Turing-complete -- at least for its domain. These are explanatory frameworks which offer a systematic perspective on setting up models of physical phenomena. Kuhn wants to say, "Yes, scientists test and throw away, like Popper says. But usually, they test and throw away at the level of this model. They don't usually throw away the theory, they just say, 'oh, this effect is non-negligible and should be included in a more detailed model.'"
Because a theory can compute anything which any other theory (in the same problem domain) can compute, there is no experiment which distinguishes the two theories. A good example is geocentrism versus heliocentrism: Newtonian mechanics fully allows you to transform to a coordinate system which places the Earth at the center of the universe with the Sun orbiting around it, by adding centrifugal and Coriolis forces. That's perfectly well allowed! You can accurately model all of the things which heliocentrism models with the Earth at the center of the solar system.
How do we choose between these? Kuhn's answers are murky, having to do with "aesthetic criteria", and I prefer an account due to Imre Lakatos. This is a feedback theory, but since one prototypical example of a feedback theory is evolution by natural selection, you might think of it instead as theory evolution or so.
What happens is, grad students are lazy and want to publish interesting results, so they select whichever theory makes it easiest for them to publish something unexpected. The Standard Model of Particle Physics is ad-hoc as all hell, and it turns out that we know that it's wrong, and it has oodles of parameters that could be tweaked to better conform with any experiment we wanted, including ways to tweak it so that there is a Higgs particle or ways to tweak it so that there is not a Higgs particle. It more or less violates all of Kuhn's aesthetic criteria. Why does anybody suffer such a thing to live? Because we can do insanely powerful calculations of real things with it, and those calculations come out correct to very high accuracy without too-complicated modelling.
Somewhat related comment:
"A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." - Max Planck
Ridley has a hidden agenda here - he's a global warming denier and he emphasizes confirmation bias to explain how all the climate scientists got it wrong (unlike him).
The cliche he derides that "scientists love proving themselves wrong" is a cliche of the climate debate to argue that the whole field of climate science is unlikely to be rotten (personally I think the point is more that scientists love proving their colleagues wrong).
There's a endnote identifying the article as "The first of three columns on the topic of confirmation bias" so I wonder if he'll do a big reveal of AGW-denial or if he's just trying to sow subtle seeds of doubt in the background.
The article conflates "science" and "scientists". Science can test hypotheses, even if individual scientists fall in love with their own theories.
The article itself gives a concrete example-- the work showing Sapir-Whorf hypothesis is a "fluke" was done by a different research team.
A more interesting question than "Do scientists suffer from confirmation bias?" might be "Are scientists more prone to confirmation bias than other professionals?" Or "In the long run of history, do the conclusions of science get overturned more often than the conclusions of other fields?"
>Science can test hypotheses, even if individual scientists fall in love with their own theories.
The scientific process is incredibly useful, but I grow tired of the near religious reverence for the nebulous "Science."
Science isn't monolithic. It isn't an object, it can't do anything. Science is merely a process practiced by people. Science is the scientists.
Science has hierarchies like any other human institution that can wrongly prevent the spread of dissenting ideas. All it takes is enough people in the right places to keep a bad idea firmly entrenched for years. As research becomes more and more specialized we may become even more vulnerable to small entrenched groups with incorrect conclusions.
Many of the incorrect theories overturned through the years, where done so, not because the scientists who make up "Science" changed there mind after seeing contradictory evidence, but because they died off.
Your comment seems to start off by agreeing with the parent but then diverges when you say:
> Science is the scientists.
> Science has hierarchies...
No, scientists have human flaws which lead to biases, hierarchies, etc. Science, as a method for discovering the truth about our world, works very well in the big scheme of things and the reverence for it is absolutely well deserved.
Whether it's true or not that older scientists sometimes have to die off before new scientists following more fruitful paths can have proper attention is a human concern; the fact is that the scientific method is ultimately very successful and the parent is quite correct in observing that the only issue here is conflating science and scientists with unfair expectations.
I don't think I remember hearing that scientist love proving themselves wrong, because they obviously don't. No one likes that. Scientists love proving other scientists wrong, which is really what this article is about. Some scientists were wrong, maybe because of confirmation bias, and other scientists without the same confirmation bias proved them wrong.
I.e., science as never been the direct route from hypothesis to proven theory that they teach us in high school.