Disclaimer: I'm a scientist, so maybe I'm biased. But then again, I'm often a vocal critic of the system myself, so maybe not.
Science is actually admirably egalitarian. That's not one of the things I think is wrong with it. The heavily politicized branches might not be egalitarian; I have no experience with them. But in general, you don't need to be a member of any professional organizations or have any other credentials to submit your work to journals and conferences (which is how we transact our business.) Many journals employ double-blind reviewing, in which case even unintentional discrimination based on credentials is unlikely or impossible.
Nevertheless, scientists might often give the impression of not wanting to let amateurs into the 'club'. Why is this? It is simply an issue of bandwidth. Think about this: for each amateur, like the author, who had something to contribute, how many do you think thought they had something to contribute but were mistaken? If you guessed a hundred thousand you'd be in the right ballpark. I've seen it in many different areas.
For example, the number of people trying to submit "proofs" of P != NP, (or worse, P = NP) is just ridiculous. Some are well-intentioned although ignorant, and others are just cranks. Some fields of research attract more amateur claims than others, but whichever field you look at, the amateurs greatly outnumber scientists, and the vast majority of them are mistaken.
So what do we do? We use a simple filter. If you speak our language, we'll listen to you. This is what appears to the lay public as anti-amateur. But it's not, really. All you have to do is to learn some simple definitions and terminology set forth in a straightforward way in previous papers, to prove that you've done your homework. Then write up what you have to say and we'll be happy to give it a read. Sure, like every heuristic, it's not perfect; sometimes there are false negatives. But there really isn't an alternative. Without a filter, all we'd ever be doing is debunking crackpot theories.
I acknowledge that much of this probably doesn't apply to climate change "science," which is a special case. But there's been a lot of criticism of science in general, and I wanted to set the record straight. It's important to distinguish between what's really broken and what appears broken, otherwise we risk throwing the baby out with the bathwater.
randomwalker is spot on, from what I've seen, e.g. in working for scientists and mathematicians. My first job was for a group of applied mathematicians at MIT (maintaining the PDP-11/44 they used to for RJE to Crays and the like, and to write up their stuff (plotting, I helped them with TeX, which was brand new at the time), etc.)
Every once in a while we'd get a "walk in", someone with something he came up with, thought was brilliant and that "the people at MIT" would be interested in. Our lead professor was quite gentle with these guys, but the outcome was clear ... and part of it was certainly that they didn't "speak our language", which when you come down to it is a pretty low bar.
Low in that reading some of the literature and sitting quietly at a few gatherings would easily bring you up to speed.
At any rate, there's not a single scientist who believes that publisher copyright for papers is a good thing. Unfortunately we can't change the system overnight. At least in CS, as I pointed out in that comment, it has already become a non-issue.
I don't think it's an orthogonal issue. I think there's genuine snobbism in academia, and the PhD system is one great part of it. The paywall is symptomatic of the fact that academics value guild credentials (PhD's, tenure, publications in prestigious journals) over equitable, open discussions.
Furthermore, people are ludicrously impressed by 'PhD's' from 'top schools' with 'impressive publication records.' Even a former academic with no institutional affiliation falls off the radar screen, and so those who want to be taken seriously in science are goaded into a costly, inefficient, and decades long rat race, just so that people will take them seriously.
You reminded me of this episode of the NPR radio show This American Life (http://www.thisamericanlife.org/Radio_Episode.aspx?episode=2...) entitled "A Little bit of Knowledge". In the third act "Sucker MC-Squared" an electrician and amateur physics enthusiast thinks he has disproved Einstein.
He finally gets an academic to meet with him to hear his ideas but won't accept it when the academic points out the flaws in his reasoning. Throughout the story the electrician insists that the math isn't important, that he's right even though he can't express the math behind his ideas. He thinks the academics are being elitist (or, as you write "anti-amateur") just because they insist on using math.
Your criticism of current scientific practice does not go far enough. Recent issues like the refusal to grant access to climate data cannot be so easily brushed off with a "not enough bandwidth" complaint.
We have the internet now. There is no reason to not put up a site with data & code when you publish your paper and just point all requests to go to the site. Scientists need to start using these tools like the rest of us and stop hiding behind such false excuses.
His disclaimer that starts with "The heavily politicized branches might not be egalitarian..." since he has no experience with them covers the particular exception you point out.
It's nice that some spend the time to clean up code and format data. But scientists progress on their career track by total number of papers; so they're being somewhat asked choose between the greater good and their own.
Most research is publicly funded. If open access is important to you then write to your government representative and ask them to require it.
The most important lesson my mentor ever taught me:
We went to a client to work on Problem X. He quickly determined that solving Problem X would achieve nothing. Problem Y was the real problem, but was way outside our areas of expertise.
So what did he do? He slept 4 hours a night for the next 2 weeks studying everything he could find about Problem Y. He reviewed reports, industry literature, called experts, and talked to as many people in the company who knew anything about that subject area. Within 2 weeks, he presented a brilliant solution that no one had ever considered but was instantly understandable by their experts. (That solution included work done by us and we had a great client relationship for years.)
Later, I asked him why he tried to accomplish something so difficult with such a seemingly tiny possibility of success. I'll never forget his reply...
"I didn't know that I couldn't do it, so I did it."
It's a charming anecdote, but don't take this as a story of individual brilliance. Read it as faith and perseverance triumphing over doubt, and conquering problems with appropriately applied work.
Note the crucial elements.
He talked to a lot of experts in field Y. This is not a "lone genius"!
He had the freedom and incentives to work on this almost uninterrupted. He would have spent at least 100 hours, which is worth at least as much as two undergraduate courses. Also, he was an expert in related field X, which must have helped.
Furthermore, he because he knew the practical problem, he could cut down the search space. He didn't have to investigate everything about field Y, just stuff applicable to him.
If you consider all that, the solution doesn't look so miraculous. What's more interesting to me is that I think even ordinary people could be achieving that kind of result all the time if they had the right background, incentives, and support.
Amateurs are a useful check on the system because they will never submit a grant proposal to the committee chaired by the co-investigator of the paper they're reviewing, and hence can make aggressive, confrontational demands of his code like running it.
My experience with reviewing papers was that it was pretty easy to figure out who the author was because each field contains a limited number of specialists. And looking at the references on a paper enables you to narrow it down.
It's a bit like the situation with date of birth and CVs: you can figure out how old someone is, if you want to know that, by looking at their degree year.
In my experience (math, physics, CS), most journals are only single blind. The referee is anonymous, the author is not.
Additionally, referees can sometimes give themselves away.
The authors also fail to cite [Irrelevant Paper, Irrelevant Paper 2, Peripherally related paper 3]. These works should be cited and compared to the work being reviewed.
"Irrelevant Paper", John Smith. Journal of Trivialities, 1995.
"Irrelevant Paper 2", by John Smith and Steve Jones. Journal of followups to irrelevant papers, 1998.
"Peripherally related paper 3", by John Smith and Phil Johnson. Journal of Blah Blah, 2002.
A minority of CS conferences are double-blind - I've only submitted to one that was. But it's difficult to maintain true author-blindness. If your work builds on previous work at all, it's difficult to maintain that. Further, I once reviewed a paper that was double-blind, and before I made it through the abstract I knew immediately where the paper came from. (Not the specific authors, but I knew it had to be from a particular company.)
Given the hyper specialisation of most academic research, circle of people involved in an area can be pretty small. You probably don't even need to read the paper - just to know what kind of approach to what problem they are taking and you can make a good guess of the people/team involved.
"The most effective people at finding errors in scientific reasearch are scientists: it was professional glaciologists, after all, who exposed the error in the IPCC 2007 case study of Himalayan glaciers." To exclude the amateur is to deny a large part of the history of science.
Well, he's not saying that a scientist is always right and an amateur is always wrong. He's just saying that a scientist is more likely to find errors in research material, which I think sounds reasonable since they're supposedly trained to do it. Then there's e.g. the good ol' "seeing what you know is there instead of what's actually there" effect which works against those who've spent a lot of time with something.
Good find on the bad data, btw! Real science is in the numbers, not in the propaganda surrounding them.
I'm pretty sure Einstein had a PhD (I believe it was on Brownian motion). Hardly a rank amateur. Patent offices don't hire Joe Soap to review patent application.
My favourite fact about Einstein is that in 1905 he submitted 5 papers - the last was his PHD submission. Einsteins two most important papers were attributed to "Mr Einstein" :)
The Patent Office wasn't paying him to do his research - he had a "day job" that he needed to do to make ends meet and to allow him some time to do what he had a passion for in his own time.
It also bears emphasizing that he was a theoretician, not an experimentalist. He made his name in 1905 explaining phenomena that no one else had been able to (or doing a lot better job of it). To do this you need familiarity with the field and time to think and so on.
So if someone with multiple doctorates resigns his tenured position and works at McDonald's flipping burgers during the day, he is back to amateur status?
I do not have my PhD yet. Yet I am clearly inside the academic system because I am at a university.
A PhD is a researcher merit badge. Obtaining it means you've been trained by people in your field how to do research. I have real difficulty calling anyone working on their PhD, or anyone who has a PhD, an "amateur."
So what are you if you are doing research and you aren't paid to do it? I've been in that situation where I had a "day" job (running a software company) and doing a PhD on the side?
I was certainly part of the academic community but I wasn't a paid "professional" researcher.
You are by no means an amateur researcher in that case. In this context, "professional" does not mean "paid to do it." It means vetted or accredited in some way. PhD students are in the process of being vetted, are a part of the system.
I guess I tend to react to the term "professional" rather strongly because I'm married to a lawyer, have a brother who is a professionally qualified engineer and I know a few medical doctors. To me those are real "professionals" - folks who have jobs where you are personally held liable and if you screw up badly enough they can take your professional qualification away from you and you are out of work as you can't legally do the job without that qualification.
In that context there is a very sharp boundary between "professional" and not - for the rest of us the boundary is rather fuzzy. I certainly don't regard myself as a professional simply because I am paid to do something or that I make my living out of it or even, unlikely as it may seem, occasionally show some competence ;-)
Yup, I guess he had a PhD from University of Zurich if I'm not mistaken. Also, he was very well aware of the cutting-edge otherwise he couldn't have "connected 'some' dots". So he was anything but an amateur, imho.
This is an important distinction, and the other amateurs always came early in the field's history when it was a simpler science.
Climate science is not in its infancy. It's broad, deep and complex.
I desperately hope you're not implying that we should give credence to the din of crackpot circus clowns who think dramatic climate change isn't really happening and that Al Gore is personally placing space heaters on the glaciers and polar icecaps.
>"I desperately hope you're not implying that we should give credence to the din of crackpot circus clowns who think dramatic climate change isn't really happening and that Al Gore is personally placing space heaters on the glaciers and polar icecaps."
No, I think we should give credence to the work of amateurs like Steve McIntyre whose high quality criticisms of specific pieces of work have been accepted into the scientific literature and who has been an important force in helping us see more clearly what is happening with the global climate.
But since you seem to have your mind made up, I guess we should just call each other names.
Climate science is broad, and a few parts are deep. Nevertheless, amateurs such as JGC and Steve McIntyre have identified and corrected errors in the field.
You are being downmodded because you are nonsensically comparing them to deranged conspiracy theorists. You do so without evidence, most likely for political purposes.
You are being downmodded because you are nonsensically comparing them to deranged conspiracy theorists. You do so without evidence, most likely for political purposes
I have no political purpose -- and I'm not sure how politics play in watching shipping lanes open up on the north pole.
I'm glad if they find errors. But correct me if I'm wrong, it hasn't change anything. And this is what concerns me: that it raises a hand for debate. The debate is over. We need to act and we need to have started yesterday. I feel like I'm watching a woman get raped, but instead of people stopping it, they want to talk about if it's really "rape" if the man wears a condom.
We are at a tipping point and we can, and should, cut carbon emissions. Moving to alternate energy sources is an important step that we will make eventually anyway, but to do it sooner means that the poorest of the world will not get quite as screwed in the end.
Most of the flaws that Steve found are in studies that claim to show that current climate warming is unprecedented in human history. These studies are important for drumming up the panic that you so ably display. It ought to be of interest to any person that we get these studies right before we squander trillions of dollars on useless environmental projects in a world where half of humanity is still desperately poor.
But it seems that many people are more interested in having their own pet ecopocalypse to feel self-righteous about than actually getting the science correct.
So you're saying the science is "wildly" off its claim? The north polar ice cap isn't already half gone? No. You want to nitpick exact numbers because that will stall any process, and hey, you have nothing to lose.
You're not poor. You're not at risk. It's the water supplies of millions of poor that are vanishing, not yours. It's the possible forced migration of millions that's pending, not yours.
These are not "useless environmental projects". They are changes we will make regardless. But doing them now will save lives.
Some amateurs have identified errors in the published literature. Because you are concerned that their corrections might affect the debate, you compare them to deranged conspiracy theorists in an effort to discredit them.
You do this because you feel that, in light of new evidence, we may not implement the specific policies you wish to be enacted. Then you claim you have no political purpose?
Some amateurs have identified errors in the published literature.
They've identified typos. If they show to be significant, I would most certainly want to know.
Because you are concerned that their corrections might affect the debate, you compare them to deranged conspiracy theorists in an effort to discredit them.
No, I simply don't want action to stop because an amatuer found a minor error and start having to go back to debating it all from the beginning again.
Amateurs scientists are still scientists, using similar methods and acting in a constructive way. If nothing else many eyes effect makes them useful - particularly in astronomy.
Journalists attempt to generate controversy to increase page views. Myles Allen was correct that corrections to climate models have come from scientists amateur or otherwise not journalists writing copy.
I agree about the journalists bit, but when he says in his letter includes the following:
John Christy took a lot of heat over the satellite data, but nothing remotely like what is being turned on Phil Jones. It would have been romantic if John's error had been uncovered by journalists combing through stolen emails, or members of the public issuing freedom of information requests. But it wasn't. It was found by the US government funding a painstaking independent analysis of the satellite record, with John's co-operation, just as Phil has said he would be happy to co-operate with an impartial and scientific re-analysis of the surface temperature record, if anyone wants to fund such a thing.
The implication is that none of these 'members of the public' are any good, the only thing worth considering is funded research.
I agree he's too broad (and in other places as well). I had assumed it was just the heat of the argument.
I also wonder about him invoking his professional capacity. It implicitly makes him a representative of scientists; in acting so he really should have been more nuanced.
Well, it used to be that being a scientist wasn't a profession at all. Most science was done by the independently wealthy.
In many fields, today, to be an amateur you need to be independently wealthy as well. Most science requires a lot of expensive equipment. The "amateurs" he spoke of were using techniques that required little money (data analysis, pen and paper math)
I'm getting tired of this "Einstein also failed a test in university" meme (usually suggesting in was in physics or math).
From what I remember from a biography he always did excellent in physics and math, and in that famous entrance exam (that I think he took a year earlier than usual) he only failed in French, which he didn't care much for.
Did I claim that? It's entirely factual to say that he didn't get into ETH Zurich, had a hard time getting a research job and did his best work while employed by the patent office.
Note that I agree with the general thesis of your article.
While you posted a fact (the failed admission test), that fact is a cherry-picked outlier that gives the opposite impression of Einsteins actual academic achievements (especially more so if you omit the rest of the circumstances).
It's like saying the fact that I won my Physics School chess championship in 1996 and leaving it there (true) without mentioning that I'm actually a mediocre player that won because many of my opponents didn't show up.
My rant was more general; I'd swear I've seen in a couple of movies the "Einstein flunked a physics test".
I'm not sure the amateur-professional dichotomy is useful.
They put the data out there and someone found an error. Someone else might spot an anomaly that leads to some important discovery.
Whether that someone is a professional scientist, a professional programmer, a statistics student or a crank is mostly irrelevant. Putting stuff out for re-use can (if planned for) be cheap and easy. There's no reason in this day and age, not to do it.
That's what's important here and pitting professional against amateur doesn't really move you forward on that issue.
I have met some people with Chartered Engineer/PE status who program - not many though. They are the only people I would call "professional programmers".
Never underestimate what a fresh set of eyeballs can do for a problem, even a highly sophisticated and nuanced one.
Good hackers work in all sorts of fields they are not experts in. I can't tell you the number of times I've seen "breakthroughs" with technical problems simply because the smart group with all the jargon had to explain themselves and their situation to the outside hackers just trying to make a computer program help them out. It's extremely easy for groupthink to take over small groups of specialists, and it happens most of the time without any of them realizing it. By the way, just so we don't feel so cocky, this also happens all the time in groups of programmers in a project who take hidden assumptions and make them into unwritten law without anybody questioning them either.
The amateur, outsider, or programmer has a vital role in technical groups and efforts of all stripes.
Science is actually admirably egalitarian. That's not one of the things I think is wrong with it. The heavily politicized branches might not be egalitarian; I have no experience with them. But in general, you don't need to be a member of any professional organizations or have any other credentials to submit your work to journals and conferences (which is how we transact our business.) Many journals employ double-blind reviewing, in which case even unintentional discrimination based on credentials is unlikely or impossible.
Nevertheless, scientists might often give the impression of not wanting to let amateurs into the 'club'. Why is this? It is simply an issue of bandwidth. Think about this: for each amateur, like the author, who had something to contribute, how many do you think thought they had something to contribute but were mistaken? If you guessed a hundred thousand you'd be in the right ballpark. I've seen it in many different areas.
For example, the number of people trying to submit "proofs" of P != NP, (or worse, P = NP) is just ridiculous. Some are well-intentioned although ignorant, and others are just cranks. Some fields of research attract more amateur claims than others, but whichever field you look at, the amateurs greatly outnumber scientists, and the vast majority of them are mistaken.
So what do we do? We use a simple filter. If you speak our language, we'll listen to you. This is what appears to the lay public as anti-amateur. But it's not, really. All you have to do is to learn some simple definitions and terminology set forth in a straightforward way in previous papers, to prove that you've done your homework. Then write up what you have to say and we'll be happy to give it a read. Sure, like every heuristic, it's not perfect; sometimes there are false negatives. But there really isn't an alternative. Without a filter, all we'd ever be doing is debunking crackpot theories.
I acknowledge that much of this probably doesn't apply to climate change "science," which is a special case. But there's been a lot of criticism of science in general, and I wanted to set the record straight. It's important to distinguish between what's really broken and what appears broken, otherwise we risk throwing the baby out with the bathwater.