> The sales and marketing departments in most companies are more powerful than their R&D departments.
Yes, this is correct.
> Hence, the design, conduct, reporting, and dissemination of this clinical evidence becomes an advertisement tool.
This does not follow well logically. The design of clinical trials still belongs to R&D and not Marketing/Sales departments. Even if Marketing/Sales want to target for a specific outcome, there is nothing saying that this particular outcome will be met until you actually try it clinically. And specific outcomes are usually desirable outcomes: for example, showing that your drug is non just superior to placebo, but superior to the current best-in-class treatment. Or measuring patient-related outcomes that are tied to economic value (for example, making a patient recover faster has tangible societal benefits - they can come back to work, be an active family member, etc...) is also relevant these days.
So, there is not a strong link between "pushing for an outcome that you can advertise" and "this outcome being clinically meaningless". If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase.
> As for basic research
Basic research should be the responsibility of academics, not companies. And they do. But not many academics focus on "checking whether what we have been using for 20 years really works", there's too much priority on new clinical targets.
> If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase.
Billions in R&D goes to getting molecular entities approved by FDA that are clinically meaningless. The medical R&D is done just to get a product that provides already existing functionality or already existing treatment with a slightly modified molecule that can be patented.
It's like the joke about man who goes to patent office and says that he invented gunpowder. Patent clerk says that unfortunately gunpowder has already been invented. The man says: "But I invented more of it." Approval process is not looking if the new drug is meaningful improvement over existing drugs. "Inventing more of the same" is approved if it can be tweaked a little to be unique.
Alternative is also true. There are potentially clinical treatments and medicines that receive no approvals because there is no profit motive to push them trough the expensive process of approval.
Drug salespeople constantly cajole doctors to prescribe one drug over the other using soft bribes like prizes, kickbacks, junkets, etc. Doctors have very limited time to do research and prescribe mostly on the basis of what sales people recommend and what patients ask for (which itself comes from drug advertisements). The "betterness" of a drug has very little to do with what doctors prescribe.
This phenomenon is quite well-documented so I won't cite sources here.
If there is once choice, doctors prescribe it. If there are two or more very similar choices, doctors prescribe one or the other. Adding competition decreases price, sometimes there may slight differences in side effects. May not be worth of the research itself, but getting into market justifies it.
Example:
Great discovery: PDE5 inhibitor helps with erectile dysfunction. Product: sildenafil (Viagra). After the mechanism was invented, tadalafil (Cialis), and vardenafil (Levitra) were pushed into market and FDA approved. There are roughly different 10 PDE5 inhibitors and the market is saturated.
Sales and Marketing do control which outcomes are allowed to be published though. If you allow me selective disclosure of trials I can show you that my roulette strategy is infallible.
As a sales and marketing person... I'll admit to a lot of blame for a lot of bad things... definitely playing with words to they point where they mean very little.
Selective publishing and "p-hacking" though, these are research/academia pathologies not marketing ones.
Study 329. GSK paid a PR company to ghostwrite a misleading paper, with the intent of concealing the fact that a) paroxetine is no more effective than placebo in the treatment of adolescent depression and b) paroxetine significantly increases the risk of suicide in adolescents.
GSK knew that their drug was worse than useless, they knew that children would die as a result, but they deliberately withheld damning data and misled clinicians. For this (and many, many other acts of deceit) they were fined a total of $3bn by the Department of Justice.
The paper has never been retracted and paroxetine is still widely prescribed to adolescents.
I understand why you are getting downvoted. Yet I think you make a valid and important point. Sure marketing and sales are putting a lot of pressure on, in that case RandD. But then RandD is, at some point, deciding to publish or find what is wanted. And, at least in my experience, good marketing people would rather not have falsified results and even if it is just for PR reasons.
Just blaming someone else, like marketing and sales, is quite often juat a lazy excuse to not have to look at oneself. Goes zhe other way round, so. Marketing saying "we didn't force them to find value X" might be literally true as marketing almost certainly didn't use these words. But pressure cam be applied in more than one way, it doesn't have to be explicit.
One good example is the series Chernobyl. Pressure from above to deliver certain things, ignoring truth until it's too late and then looking for someone to blame. Plant management did it with the reactor control crew and the political higher ups indirectly with plant management. So yes, all of them have to blamed. Not simply a communist problem as this thread about pharma R and D shows, right?
Well said. In fairness to the downvotes, I was bey tongue-in-cheek.
Whether for reasons of communist 5-year plans, capitalist pharmaceutical industries or even just the personal desires of an academic to prove a pet theory, publish or whatnot.. incentives and pressure to cheat exist. That's one reason why replication is so important.
> Hasn't this been fixed with the requirement to pre-register trials?
If it was that simple...
Preregistration is a step in the right direction, but there are still plenty of ways to game the system, particularly as it's not properly enforced: http://compare-trials.org/
But it is the necessary step to highlight the problem so that actions can be taken.
Also, you can safely assume that the large majority of outcomes that were not published were:\
- negative
- worse than negative (i.e. safety issues?)
- not properly designed
So if you compare the number of trials disclosed/not disclosed for each drug/each company that gives you a pretty good picture of who is doing what already.
With that data, it's then a lot easier to approach regulators to do something about it.
My assumption is that most unpublished trials were "no effect".
Be a drug company. Develop a "drug" that is really a placebo (has no effect). Do 10 trials. 8 are "no effect", 1 is positive (by chance), 1 is negative (by cance). Publish the positive drug. Market the drug as having positive effect. Sell the drug, make billions. It would take decades for enough other research (mostly "no effect") to accumulate before it becomes obvious that your drug actually has no effect.
That’s the whole point of preregistration. The nine unpublished trials can be assumed to be negative, or at least neutral, and were aware that they were done because of preregistration. Therefore regulators have the ability to reject the notion of the effectiveness because so few of the studies reported positive outcomes. In addition, any cancelled studies now taint the authors and sponsors with a (small) amount of suspicion that they are refusing to publish non-positive effects which, cumulatively, will cause their results to be further discounted. That will hopefully provide a strong incentive to publish neutral and negative results.
A drug that genuinely has no effect is actually pretty good; most drugs have some sort of risk and many are seriously dangerous. Rofecoxib is believed to have caused around 100,000 heart attacks and 30,000 deaths. Market approval for rofecoxib was based on misleading data and its withdrawal from the market was substantially delayed by misleading data.
From the results there it is clear that they failed to game the system since the gaming was detected. Getting to the point where regulators can actively use the preregistration information is the next step, but the foundation is there.
This really isn't true, certainly not as a blanket statement.
What Ioannidis is saying is that profit motivates pharma research. This is in a sense a tautology. Even then, Pharma do fund trials initiated by investigators not affiliated with the company.
Until recently, Big Pharma CEOs were either medical doctors of pharmacists. Marketing and finance were important, but not a driving force. Nowadays all of Big Pharma is headed by bean counters, marketing drones and finance guys, and are only interested in EBITDA. This is the main problem in all industries were science and engineering once prevailed.
> In 1997, when Dr. Andrew Goudie, a psychopharmacologist at the University of Liverpool, asked AstraZeneca to fund a research study he was planning, a company official replied that “R&D is no longer responsible for Seroquel research—it is now the responsibility of Sales and Marketing.” The official also noted that funding decisions would depend on whether the study was likely to show a “competitive advantage for Seroquel.”
This isn't a small company nor is it a minor drug. This is one of the largest drug companies running a trial on one of the most lucrative drugs in the world.
And secondly, why should ANY drug trial be managed by a PR department? When you ask people to risk their lives to be in a drug trial (in the link above the subject died in the trial), why should it be to improve the PR of a drug for one company. The fact that this exists at all is a major warning sign. The indifference of people to this is frankly amazing to me (its even worse than the disinformation provided by the OP).
Instead of trying to reason from first principles about whether or not what he's saying is true, maybe instead go on Amazon and buy one of the dozens of books on the subject.
I don't know if what he is saying is true, but I do know from having worked in another industry that the idea of sales/marketing dictating product development is not necessarily illogical, even if at face value it is.
> If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase.
Go look up how many corporate stooges are in the place in the FDA over the years.
A good starting point is the history of Aspartame, I'd link some articles, but I suspect the sources would be shot down and evidence ignored. The people at the FDA were complicit in its approval, despite scientific evidence showing it was dangerous.
Aspartame is ironically a substance we are most sure of its safety because of how fixed many were on the idea it was dangerous. Few substances were given such exhaustive testing.
The sources would be shot down for a reason and not some vast conspiracy but because the hill they chose to die on wasn't even real.
I would draw the opposite conclusion - safety analysis by whatever becomes a cause is a very poor way to both allocate resources.
Err, have you acually read the studies they used to approve? They are not unequivocal, and the final approval of it as a drink additive was thanks to Donald Rumsfeld, who was CEO of Searle at the time and had a hand in selecting Arthur Hull Hayes Jr. to head the FDA.
> I'd link some articles, but I suspect the sources would be shot down and evidence ignored
Why would you say that? This is a serious accusation made without offering evidence and the only effect it can have is to greatly diminish the probability of having a productive debate.
I suspect that the sort of person who would be convinced to look it up and decide for themselves would be less likely to do so after reading the sort of comment I replied to.
Assuming the FDA is corrupted is one thing, assuming regulatory bodies ACROSS the world are all corrupted at the very same time is just very close to being a ridiculous claim.
There is a big difference between what law-makers/legislators think is safe in the context of their desire to promote the economies of their commercial partners, and what scientists consider safe in the context of human health and safety.
The two cannot be conflated.
Scientists can call for something to be re-considered, safety-wise, long after legislators have put their deals to bed. There are countless examples of this throughout history, where researchers, engineers, scientists have had to struggle very hard to get politicians - who have their own interests - to address the issue.
Too many times, research which runs counter to economic narratives is ignored or, worse, suppressed in order that people can 'continue living their lives as consumers unhindered'.
Suggesting that people bow to the authority of legislators over that of human-health scientists serves only to perpetuate the conditions which endanger us all.
Remember, asbestos was once 'scientifically and legally proven to be safe' at one point. Too many times in history, what the collective thinks is safe is only so, because the collective ("everyone thinks the same") says so ..
> As far as artificial sweeteners go, you'd have to be an idiot to knowingly consume an unnatural, laboratory made chemical.
Don't worry, I stripped the <natural> XML tags from regular sugar and mixed them up with my aspartame; it cheats reality into thinking that the sweetener is natural, preventing it from hurting me.
Authorities are called such for a reason. It is only a fallacy to invoke an authority who is unrelated to the subject, or if your opponent is also an authority.
> As far as artificial sweeteners go, you'd have to be an idiot to knowingly consume an unnatural, laboratory made chemical.
On the subject of fallacies, this one is called the “appeal to nature”.
"Not infallible" doesn't mean "always wrong". Just because FDA made some mistakes, doesn't mean you can start doing the opposite of everything they say and live a happy and healthy life.
Authorities may be "in the pocket of Big Pharma", but by default, I'd still trust them much more than yours or mine non-expert opinion.
Authorities don’t have to be infallible to have a better track record than anyone else. If they did, we wouldn’t have senior software engineers, we would have the pointy head boss writing all the code; we wouldn’t have doctors, as everybody’s home remedy would be just as good; we wouldn’t have CEOs, as communism would be correct and the workers would be more effective at running their businesses than the owners ever were (hello ironic counterpoint); etc.
Of course, everybody including authorities is capable of making mistakes. It is generally a bigger mistake to think you know better than they do. Not always, but generally.
It's way worse than that. Fluoroquinolone antibiotics are well known in research for being seriously dangerous yet they were prescribed like candy for over a decade. So many people develop disabling symptoms overtime that end up not being linked to the drug because dna adduction and mtdna depletion won't necessarily show up quickly. The drugs have likely disabled hundreds of thousands at this point by Dr Bennett's estimations at UNC. There's something like 10,000 reports and those are assumed at being less than 10% and possibly 1% of total incidents.
He is a spiritual brother of Dr. Peter Gøtzsche, who is also very clearly describing the corruption going on in the medical sciences.
He made simple statistical analysis to prove certain drugs like antidepressants causing side effects / death. His findings ignored, he was removed from his leading position from Cochrane Collaboration.
He does meta studies -- aka studies of studies -- and finds that most studies are flawed, so most of the evidence we think we have isn't really there.
It doesn't exactly make him popular because he calls into question a lot of existing supposedly "proven" medical science.
He talks a lot about the biased assumptions that are the foundation of a lot of the framing of studies. If your baseline framing is sufficiently flawed, the entire study is junk. He thinks a lot of studies are, in fact, junk.
Stupid question - what makes us think the meta studies are themselves valid?
Also wouldn't junk itself have many subclassifications as how it is junk? Say poor study data quality, wrong assumptions of wider applicability, wrong analysis or conclusions, etc.
what makes us think the meta studies are themselves valid?
A. This is one guy, basically. I mean, I agree with him, generally, but not everyone does.
B. It's easier to poke holes in someone else's thing than create a good study.
So study A: "I wonder how fat you have to be to not fit through this door. Let's weigh people (instead of measuring them) to see how fat you have to be not fit through the door."
Study B: "It's a great question, but it's badly done. Weighing them won't work. Let's just interview people and write down their self reported height!"
Meta analyst: "Y'all are both wrong and stupid, geez."
Scientists A and B: "Let's take the meta-analyst out back and beat him up!" "Sounds good to me!"
I'm sure the people doing the meta studies would be delighted to have other, independent people attempt to replicate their studies, but as they did it, and in other ways to verify their findings. Unlike many of the groups that did the studies this meta is studying, who often don't want any group do anything except confirm their findings because of the money and status behind their studies.
>> Stupid question - what makes us think the meta studies are themselves valid?
This is not a stupid question. Many meta studies are done by people with their own frameworks of deciding what is and isn't a legitimate approach to studying something. Unsurprisingly, their findings are overwhelmingly negative due to their own publication bias.
Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data, and any study that received - directly or indirectly - more than $1 from the public taxpayer should be forced to publish Open Access.
Lay bare all the raw data and let the software developers and data analysts get after it, not so-called expert meta reviewers.
> Lay bare all the raw data and let the software developers ...
Really can't tell if you're being sarcastic here, even in the context of HN.
> Meta studies are pretty worthless in my opinion. Primary studies in all fields should be forced to be published with open data ...
And until that happens meta studies are both worthwhile and one of the most effective ways to correlate and sensibly interpret the existing published data sets.
I read the J Clin Epi paper being discussed when it first came out and it expressed my feelings and thoughts very well, as a professor.
The crux of the problem is this: we as a society value science, and the epitome of this in healthcare has been the goal of evidence based medicine (EBM) or evidence based practice (EBP). The idea seems simple enough, that healthcare decisions and procedures should be based on scientific evidence.
This clashes, however, with the reality that much of research in the biomedical sciences is unreplicable or completely invalid for other reasons. The reasons for this are complex but you have things like p-hacking, selective reporting of results, confounds, and so forth and so on. Think "how to lie with science" instead of "how to lie with statistics."
As a result, EBM gets hijacked by those who are trying to manipulate; appeals to science becomes a form of fallacious argument, sort of a version of appeal to authority error, where one appeals to a scientific finding because it's peer reviewed etc. without invoking the underlying integrity of the research per se. Then you have the unscrupulous making money off of bad science, and authority figures (in medical care and administration) cudgeling people for not using EBM/EBP when what is really happening is people are recognizing it as bad science. Sometimes no science is better than bad science--or sometimes, science means rejecting peer-reviewed research, even with all the right surface features.
The paper is part of a slightly personal exchange between Ioannidis and someone he looks up to but is also close to, so it has a kind of poignant conversational style. It's not so much a scientific paper as it is a personal discussion.
Dubious evidence in medical studies is a big problem.
But I'd argue that it's only the core of the issue. The second layer out — and I think the more influential layer — is the JOURNALISM surrounding these medical studies.
Even if the underlying science is sound, a lot of outlets and reporters routinely (and sometimes willfully) get the facts wrong.
I attended a talk with Retraction Watch's Ivan Oransky, where he spelled out how easy it is for well-intentioned journalists to exaggerate, over-generalize, or misunderstand what a study says. It was eye-opening. And a little frightening.
CDOs and the subprime mortgage crisis have some analogues to pre-Enlightenment darkness in human intellectual history. The idea that authoritative figures can use maths to give junk debt the midas touch is basically financial alchemy.
There was no sound reasoning behind it. It was literally just "maths is really powerful, and these people are the Smartest Guys in the Room, which is not really that much different from "these guys are really Holy, and also the Smartest Guys in the Room" in previous generations of humanity. Just market it with Math, and you can defraud the most powerful and prosperous society in history.
After reading the interview, it sounds like a lot of scare mongering about bias without actually identifying specific things that are wrong with specific examples. Sure, it's nice to be looking for these issues, but people obviously have different opinions on what biases produce poor results and if biases are actually necessary and helpful.
Yes, this is correct.
> Hence, the design, conduct, reporting, and dissemination of this clinical evidence becomes an advertisement tool.
This does not follow well logically. The design of clinical trials still belongs to R&D and not Marketing/Sales departments. Even if Marketing/Sales want to target for a specific outcome, there is nothing saying that this particular outcome will be met until you actually try it clinically. And specific outcomes are usually desirable outcomes: for example, showing that your drug is non just superior to placebo, but superior to the current best-in-class treatment. Or measuring patient-related outcomes that are tied to economic value (for example, making a patient recover faster has tangible societal benefits - they can come back to work, be an active family member, etc...) is also relevant these days.
So, there is not a strong link between "pushing for an outcome that you can advertise" and "this outcome being clinically meaningless". If it were meaningless, FDAs and other regulatory bodies would not even accept it during the clinical trial design review phase.
> As for basic research
Basic research should be the responsibility of academics, not companies. And they do. But not many academics focus on "checking whether what we have been using for 20 years really works", there's too much priority on new clinical targets.