The Evidence Against Evidence-Based Medicine

A worthy goal built on a weak foundation

The practice of Medicine is moving in the direction of being more “evidence-based.” The goal is to provide improved patient outcomes. The field of Nutrition is also following the lead of the Medical field in becoming evidence-based. This is all great in concept. The foundation of this approach is to use the best available clinical evidence for patient treatment and guidelines. Unfortunately, the foundational base of evidence is filled with bias. As such, the foundation of evidence is crumbling.

It’s hard to trace just where it all started to go wrong — but I’d bet money on it being about money. At the headwaters of the river of money that is eroding the foundation of evidence are the Pharmaceutical and Food industries: That’s where the flow of money starts. And there is a lot of money to go around and a lot of bias as a result.

How can this happen if treatment and guidelines are based on clinical evidence?

To explain how, we need to look at how clinical evidence comes to exist and then how it becomes known in the Medical and Nutrition communities.

Clinical evidence is the result of research conducted by scientists. These scientists work in labs associated with a university or a hospital setting, or, in a private sector facility in a pharmaceutical company or food company. Researchers in the university and in some hospital settings need to publish clinical findings to either keep their job or to advance their career. These researchers need money to fund their research and are always searching for sources of money. While there are grants available from the Federal and State governments, there isn’t always enough available from these sources, and Big Pharma and Big Food are happy to step in and help.

This financial help often comes with a voice from the sponsoring company or association that influences the research design of the study — how many people are tested, how long is the treatment, etc. A skilled researcher can sway the chances of getting a desired result in their favor — but they can’t be obvious about it, because the journals they want to get the results published in are typically peer-reviewed. But it can be done. See my article on how a low-carb diet is often tested in research designed to show the diet isn’t so helpful.

But does this really impact the “best available clinical evidence?”

Let’s look at a review of studies that looked at the association between drinking sugary beverages and weight gain [1]. The cited reference is actually a review of several reviews with an eye toward financial conflicts of interest within the studies. The review of reviews looks at both Randomly Controlled Trials (RCTs — which are the strongest research design) and observational studies. But here, we’ll just look at the results from the RCTs.

A total of 59 RCTs were included in the review. Of the 59 studies, 49% shows a positive link between drinking sugary beverages and weight gain: the more sugary beverages that were consumed, the more weight was gained at the end of the study. The other 51% did not show this positive link. These results (shown below) would tend to suggest that there are other factors instead of, or maybe in addition to, drinking sugary beverages impacting weight gain.

But look what happens when you split out the studies where there were no conflicts of interest versus from those where conflicts of interest existed (see below). I think it’s clear that there is a bias at play here. The problem is that these conflicts of interest aren’t always disclosed — and when they are, they aren’t necessarily put front and center. With some online journals, you need to click a link labeled either About the author(s) or Disclosures to see any conflicts.

If that was the only source of bias wearing away the foundation of evidence-based medicine and evidence-based nutrition, it wouldn’t be so hard to fix. Making conflict of interest disclosures more visible would be one way to make our best available clinical evidence at least more transparent if not bias-free. But it gets worse.

What gets published versus what doesn’t get published has a big influence on the evidence base doctors and nutritionists have to develop treatments and guidelines around. There are many influencing factors that impact what gets published. First, the researchers tend to know what kind of results specific journals are looking for. In general, journals want exciting, positive results (I’ll discuss why in a bit). If it looks like a drug trial isn’t panning out, a researcher might decide to stop the trial early to save money, given it wouldn’t likely get published anyway. This happens even when Big Pharma isn’t funding the study. If Big Pharma is funding the study, they can spike the results by not allowing the study to be submitted for publication by the researchers.

Second, the Editor-in-Chief of a journal wants to publish exciting, positive results. When this type of article is published, the sponsor (if it’s a drug company) buys thousands of reprints of the article to hand out to doctors. Those reprints are big business for these journals. Drug companies don’t buy reprints for drugs that fail their clinical trial. So journals tend not to publish those. Journals also stay away from studies that “fail to replicate” — where a second research team copies the design of a previous exciting study, but they get different (non-positive) results. With these, in addition to sponsors not buying reprints, the journal editorial board doesn’t want to show that the exciting study they published several months ago wasn’t so exciting and positive after all. Also, publishing exciting results gives the journal more citations which improves its impact status, which will get more researchers trying to publish with the journal, and more chances of finding additional exciting results to publish.

But what about peer review?

The peer review process is managed by Editors. Editors are the gate-keepers of what gets reviewed for publication and what does not. They can refuse to review articles that contain research results that they don’t like — and if the article doesn’t get reviewed, it doesn’t get published. So, the Editors hold a lot of power. Big Pharma and Big Food try to influence that power in their favor. In a 2017 report in the British Journal of Medicine, it shows that Editors of the Journal of the American College of Cardiologists received industry payments of over $475,000 — EACH. [2] I imagine that buys a lot of selective reviewing on the part of the Editors.

These Editors are providing the framework for the evidence-based foundation doctors and nutritionists are looking to when they develop their treatments and guidelines. But the whole picture isn’t being shown.

For example, the published literature on antidepressants looks extremely positive — with 38 out of 41 clinical trials (94%) showing positive outcomes for users of the various drugs. But in January of 2008, the New England Journal of Medicine published a special article on selective publication with respect to antidepressants. [3] Of the 74 antidepressant drug trials that were registered with the FDA, 23 were never published — all of which had “non-positive” results. When the published articles were reviewed by the FDA, they found several had non-positive results, but were written up as if the result was positive (see my article on relative risk versus absolute risk). When these “spun” articles and the non-published results were taken into account, the results show that only 51% of studies had a positive result.

How are doctors and/or nutritionists supposed to use the best available clinical evidence to base their decisions on when much of the evidence they have available is biased by the pharmaceutical and food industries?

What are your thoughts?

Thank you for reading this article — hopefully it contained something you found useful.

If you aren’t a member of Medium but are thinking of joining, please join through my page! If you do sign up to Medium through my page, some of your membership fee goes to me (but you still pay the normal membership price).

With a paid membership to Medium, you will get to read more of my work plus you get unlimited access to thousands of Medium writers. And it’s only about $5.00 a month!

I wrote a book about nutrition, metabolism and weight management. It covers those topics in depth — but not so much that you need an advanced degree to understand it! See my profile page for more details, if you are interested. It’s available on Amazon.

References

1. Bes-Rastrollo M, Schulze MB, Ruiz-Canela M, Martinez-Gonzalez MA (2013) Financial Conflicts of Interest and Reporting Bias Regarding the Association between Sugar-Sweetened Beverages and Weight Gain: A Systematic Review of Systematic Reviews. PLOS Medicine 10(12): e1001578. https://doi.org/10.1371/journal.pmed.1001578

2. Payments by US pharmaceutical and medical device manufacturers to US medical journal editors: retrospective observational study BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j4619 (Published 26 October 2017)

3. H. Turner, M.D., Annette M. Matthews, M.D., Eftihia Linardatos, B.S., Robert A. Tell, L.C.S.W., and Robert Rosenthal, Ph.D. Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy, New England Journal of Medicine 2008; 358:252–260