Hacker News new | past | comments | ask | show | jobs | submit login
ML researcher replies to request to review paper for closed journal (plus.google.com)
209 points by robrenaud on Jan 18, 2013 | hide | past | favorite | 25 comments



The earliest really large-scale instance of this occurred in 2001, when more than 30,000 researchers signed a letter boycotting any journal which didn't make articles open access within 6 months of publication. The boycott meant not reviewing for, editing for, or publishing in, any such journal. Signatories included many of the best-known scientists in the world. One of the authors of the boycott was Nobel Laureate Harold Varmus, who had just completed a term as Director of the US National Institutes of Health.

It's unclear how many of those researchers followed through on the boycott. Some did -- I've met many scientists who say they send letters similar to LeCun's. I've also met some who signed, but who now admit (often sheepishly) that they didn't follow through.

Text of the letter is here:

http://www.plos.org/about/what-is-plos/early-history/


I feel that it's important enough to reiterate that PLOS is a non-profit organisation and relies on donations to keep it going:

http://www.plos.org/support-us/individual-membership/#make-a...


Huh? They actually charge a hefty fee for publishing: http://www.plos.org/publish/pricing-policy/publication-fees/

The donations and open-access merchandise are a nice extra.

They are turning fabulous profits (almost $3 million in 2010; $4 million in 2011 -- these are net profits), but, being a non-profit, must invest them back in the organization.


The machine learning community has been pushing for open access for quite awhile.

In 2001, the 40 editors of Machine Learning (some of whom I was quite proud to know), resigned to start the JMLR with this letter: http://www.sigir.org/forum/F2001/sigirFall01Letters.html . Note also their citation of the similar actions taken by the editors of Logic Programming in 1999.

EDIT: btw, the JMLR is at http://jmlr.csail.mit.edu/papers/ . moar edit: http://jmlr.csail.mit.edu/mloss/ for relevant open source code.


JMLR is an amazing success -- a great example of a very high quality open access journal that doesn't ask authors to pay any article charges.


It's worth noting that Cosma Shalizi has been doing this for about six years now, and it's becoming increasingly popular in certain areas of academia.

Background:

http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/442.html

http://masi.cscs.lsa.umich.edu/~crshalizi/weblog/864.html

http://gowers.wordpress.com/2012/01/21/elsevier-my-part-in-i...


Terry Tao made a good point when he joined the boycott of Elsevier journals. He said he would stop submitting to their journals, and stop doing editorial work for them, but declined to stop reviewing papers.

The reason he gave is that while he is at a stage in his career where there is no pressure to publish in any particular set of journals and so can exercise discretion in where he publishes, he recognizes that some mathematicians, for reasons beyond their control, are unable to do so, and he does not wish to penalize them.

For fields where many of the high impact journals are not yet open access, I think Tao's approach makes sense. Publishing in high impact journals greatly helps the careers of young researchers.


Weird. Terry Tao posting a note on his Google+ feed saying "this is a good paper", with few relevant +1s, should be more of a resume booster than any journal publication.


In my opinion, Yann LeCun is one of the top celebrities in ML/Neural Net research. His voice really has weight.


Definitely. At work today we were watching a video on Machine Learning by Andrew Ng and he made a number of references to LeCun's involvement the field, and listed him in the acknowledgements section of his talk. It has encouraged me to look into more of his areas of work.

I signed up for Andrew's ML course (https://class.coursera.org/ml/lecture/preview , start date is not announced yet). Really looking forward to it.


I took the very first course when it was a Stanford experiment and I have to say that was a lot of fun! It feels a bit downgraded in terms of difficulty but quality wise was very good.


I heard about him from his Google tech talk on learning visual models. Interesting guy.

http://www.youtube.com/watch?v=3boKlkPBckA


I did a course this semester at university about bioinformatics. I was very impressed by their attitude to knowledge sharing. Almost every paper referenced by our professor and any citations in those papers that I used were published online for no charge. But even better than that, the data and code used to generate their results was shared too. Our professor noted that in the bioinformatics field a paper is considered pretty much worthless if the data isn't made easily available in one of a few archives.

I really hope the rest of acadaemia can follow through. It seems to be we might make more progress in all of the sciences if results are easily replicable and research is published to all.


This is a very forthright reply by the researcher asked to review the paper. Right now, reviewing submissions to scientific journals is anonymous, and not well rewarded. Jelte Wicherts, writing in Frontiers of Computational Neuroscience, (an open-access journal) suggests new procedures

Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci., 03 April 2012 doi: 10.3389/fncom.2012.00020

http://www.frontiersin.org/Computational_Neuroscience/10.338...

for making the peer-review process in scientific publishing more rewarding and more reliable too. Wicherts does a lot of research on this issue to try to reduce the number of dubious publications in his main discipline, the psychology of human intelligence.

"With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses."


> I have pledged to no longer do any volunteer work, including reviewing, for non-open access publications, which unfortunately includes pretty much every publication from [commercial-publisher].

The volunteer aspect of peer review is the worst part. How much is an expert's time worth? How much is a two-sentence review worth to an author? If it were paid on an hourly basis using an honor system, it would be fantastic for quality and consistency.

That closed journals ask for volunteer review makes it doubly bad. But I still believe paid review for open access journals would be a good thing. I know, it is an unrealistic fantasy.


Speaking as an author, paper reviews are very helpful (although the useful ones are generally much more than two sentences). Speaking as a reviewer, I don't mind doing reviews for free (I consider it part of the job) and if volunteer reviews are needed to make open access work I think it's a worthy tradeoff.


Professors and researchers should start charging for time spent helping the closed journals at the very least.

If you think your reviews are valuable, charge accordingly. You provide value to the publishing process and should capture that value. You are providing the service of "expertise". If you provided that service to a lawyer, you would earn several hundred dollars an hour.

There's probably a business opportunity in creating a clearinghouse for reviewers, editors, etc. and managing the accounting.

Edit: Grammaring more goodly


Professors are the producers and consumers of these journals. So, a group of your peers invests time reviewing your work and giving you helpful feedback, and you pay them back by reviewing other people's work.

Basically, it's a zero sum game. So if everybody paid for the reviews they did and then got paid for doing reviews, a lot of money would change hands and everybody would end up with the same amount of money they had to start with.

Of course, the companies that _publish_ the journals take a cut. But that's what it is, a cut. There are really communities of professors participating in a zero sum game.


I beg to differ a little, Professors should be paid to do reviews, and not pay to have reviews done.

The companies that publish the journals don't take a cut, they take the lion's share basically, paying a professor to review would work better in having them take a share of the pie.


But if you pay professors, you either pay them enough to make it worth their time, or you pay them less than that.

In the former case, you are incentivizing people to do reviews who are not actually qualified to do them. Or, incentivizing people to do more reviews than they have time to do a good job on.

In the latter case, there's not really any point.


This is a great first step. But for this to really make a difference to the current traditional publishers - every researcher would need to follow this example. The other problem is access to all the previously published papers. So my question is : Who owns the copyright to a paper published in a major academic journal? Is it the original authors / researchers? One would think so since they are the authors. Or do they grant exclusive rights to the journals. If the authors own the copyrights, what is to stop them from uploading their papers to one of these Open access sites?


Typically the publisher retains copyright, they make the author(s) sign it away as a condition of publication. There are some exceptions.

By inserting themselves into the somewhat "broken" system of scientific publishing, the large publishers have been making a killing. (It's broken from an economic point of view because too much work is done for free.)


I really don't want to derail the thread, but one of the people participating in that G+-thread (Mike Taylor) has written some very good articles at http://reprog.wordpress.com/ (and I really can't remember if I picked up the subscription here or elsewhere)


I would love it if somebody could register a domain and create a canned reply that we could copy paste when asked to review papers with appropriate links to all relevant open access projects.


Who is that guy saying that publishing in arxiv is career suicide? Why are there ML people playing paper chase at universities when they can quickly make a fortune in industry and retire to a life of academic leisure. Of all people, ML researchers should have noticed that the 21st century has arrived.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: