I like his rant. But at the same time, as an ex-CS grad student now doing biology, I wish my field was 1/10th as rigorous, tidy and beautiful as computer science.
Code is a joke, most data is processed using "pipelines", which in reality means some irreproducible mess. People don't generally do research trying to understand how cells or tissues work, they generally write papers about "stories" they found. Only a small minority are trying to do some serious modeling using serious math.
> Code is a joke, most data is processed using "pipelines", which in reality means some irreproducible mess.
You're not wrong, and it's not limited to bioinformatics; Reinhart-Rogoff's findings were reversed when an additional 5 rows were included in a spreadsheet they used to calculate their correlation between GDP growth and debt ratios. And of course, they insist that despite the actual outcome being twice as strong and in the opposite direction, they still support their original position.
I wonder if one can get a CS PhD by producing enough retractions. Of course, it won't win you many friends in the academy, and would probably lead to less source code made available. But given the Perl code I've seen published who's termination condition is a divide-by-zero exception, one can argue that peer review in the information age has to include code review.
Didn't know about the Reinhart-Rogoff controversy [0], interesting! They state that they have been careful not to claim that high debt causes slow growth, but rather that it has an “association” with slow growth.
I read abt this quite a bit, the refutation is attacking a small portion of the data, because that small portion is trendy and hot in politics.
Judging academically, the original paper and the refuting paper is a healthy debate, but the dynamic of the society and politics ab-use them to attack a whole school of thought at large (the austrian school: less bailout, less intervention by government, less control over everything) in favor of keynesian school (more bailout, more government spending, more public debt, especially in recession and crisis).
Anyway, it remains a controversy, because theoretically one can do what one wants, but once it involves policy and real life matters, it is hard to argue for what method is right and what is wrong, in the presence of so many (ready to b angry) interest groups.
Excuse me? That single attacked paper was the intellectual blanket for an unprecedented victory march of the Austrian school after the financial crisis and the recession.
I agree that from a purely academic point of view this is nothing big to worry about, but this paper played a completely outsized role. And the authors stood by and let things run their course, without any attempt to reign in or moderate the debate.
Fair enough, but i add that academic life is sad, one has to pursue one's endeavor at one's own cost. However, politicians and the public want too much from us researchers. So sometimes, we do believe that our sweating formulas have life impact, or to fancy, save the world.
>>People don't generally do research trying to understand how cells or tissues work, they generally write papers about "stories" they found. Only a small minority are trying to do some serious modeling using serious math.
Speaking as a layman, that seems like a very strong claim for what one hopes is a hard science supposedly applying the very best practices of the scientific method (i.e. falsifiable theories vs. anecdotal stories).
Is this meant to be hyperbole to get your point across, or something that is generally known throughout the bio[med|tech] industry? As a sister comment pointed out, the latter scenario would be quite alarming.
EDIT: I'm aware of growing sentiment within the scientific community to reconsider using p-values, of which John P.A. Ioannidis and his body of work[1] helped to raise awareness of. Was this the "story"-like theme that you're referring to in cell and tissue papers?
His claim is hyperbolic, but only from the perspective of the way that scientific research has been conducted for the last 10-15 years. Prior to this, many researchers were forced to tell "a story" because the sheer volume of data and complexity in fields like cellular biology was/is so massive. This does not mean that their findings were in any way intentionally deceptive, it's simply that it was impossible to cover all the ground necessary to tell the whole story.
However, as data analysis techniques have become more powerful and data more easy to produce thanks to advances in the scientific methodologies we use to access information about the matter we are studying, the datasets are getting bigger but also easier to manage.
Biology is driven by genetics, and it's what grounds the field. I think that, perhaps, your experience is more in bioinformatics, for which I share some of your skepticism. Some good work is being done there, but usually that has a strong genetic component.
We're at an interesting point where many of the useful mutant screens have been done to saturation, at least for the most common species. Expanding our genomic resources to screen more species is certainly helping. The way forward is just the same as it ever was; good mutant phenotypes with strong genetics.
I don't think 'serious math' is at all useful for most biological investigation. Modelling is only a means for generating hypotheses, and in my experience it's a terribly weak tool. A poor substitution for proper genetics.
I mean really, the Churchill quote about democracy applies here as well: the current way of doing research is the worst possible way of doing it, except for all the other ways we've tried before.
VUSes that actually could be clinically significant are hoarded, and conversely, some companies claim to know that certain VUSes are not based on their own proprietary data that no one else sees/double checks. meanwhile the actual data comes from patients, who are left in the dark
That's just cruel. Comparatively, at least CS gets math formulations
Code is a joke, most data is processed using "pipelines", which in reality means some irreproducible mess. People don't generally do research trying to understand how cells or tissues work, they generally write papers about "stories" they found. Only a small minority are trying to do some serious modeling using serious math.