avatarMarmotian

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

6066

Abstract

on’t secure research funding first. It’s not hard to imagine then a small percentage of researchers would fake/sensationalize how promising and “novel” their research is, by manipulating words, data, or statistics, or backstab rival ideas and theories by misrepresenting them. Similarly, for tech or application research, it is necessary to create an apparent appeal of their products in the market — science sells. These motivations coupled with the advancement of financial technologies that can move money around quickly have resulted in immense pressure of hype and fraud creeping into every corner of scientific research, arguably much more pervasive than in the old times.</p><p id="94ca">Admittedly, the reliability of scientific knowledge is easily undermined in such a research environment. Scientific research bears a <b>“compounding” reputational effect</b>, in that if one has successfully gotten funding for a particular type of research, they are more likely to keep the money (funding) pumping by continuing to further develop this original narrative (rather than ditching it and changing research direction). Same in paper publication, <b>after having gone through the peer-review process, statements/knowledge embedded in a paper are automatically taken as some kind of serious “truths”, and its dissemination runs almost purely on trust.</b> That is, under publish-or-perish, even a slight misrepresentation in the beginning can snowball into a whole field of problematic research due to the obsession to maintain funding and fame (some countries even have <a href="https://www.nature.com/articles/d41586-018-06185-8">cash reward</a>s for every paper published!). Research success now comes from getting past a one-step initial gatekeeping and subsequent high <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0073381">output numbers</a>, less the actual quality of science.</p><p id="e4dd"><a href="https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease"><b><i>Extra</i></b></a><i>: Fraudulency in “influential” Alzheimer’s studies has spawned an entire field of questionable research</i></p><h2 id="d7d3">“Rigged” research structure</h2><p id="a1f8">Money manipulates. A capitalist-minded funder of course has a tendency to seek those who can write a good story/proposal with the appeal of a “silver bullet”. Frankly, in a dog-eat-dog economy, who would fund a 5-decade research program that does not promise to deliver a clear, attractive goal? Researchers either have to break the program down into small pieces (aka. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3999612/">salami slicing</a>) or else they would have to sensationalize its attractiveness, the <a href="https://www.youtube.com/watch?v=LJ4W1g-6JiY">PR surrounding nuclear fusion</a> research is a good example in action. There is this ubiquitous mindset that the more and quicker you invest in a particular “attractive” research, the faster the next “breakthrough” will come, irrespective of what’s going on in the rest of the world.</p><p id="f9c4">It is no coincidence that there is a glaring pervasiveness of short-term, small sample-size research plaguing all of Science nowadays. Such handy programs are attractive because they are methodologically less standardized (i.e., more easily manipulated via <a href="https://www.vox.com/science-and-health/2017/7/31/16021654/p-values-statistical-significance-redefine-0005">p-hacking</a> to achieve “<a href="https://www.nature.com/articles/d41586-019-00857-9">statistical significance</a>”) and can produce quick deliverables (i.e., the illusion of something being “done”), despite being unable to generate reliable knowledge and insight. Unfortunately, science is never about “attractiveness”, it is about continuous testing and <a href="https://www.nature.com/articles/533452a">reproducibility</a>, refuting or refining previous theories, however “boring” it may sound. Every researcher probably understood this at some level in the back of their mind, but the drive of capitalism is one hell of a drug that promotes mind-body dissonance.</p><p id="4b4d">This is not to say that individual miniature studies are necessarily worthless, but they only work if what you are concerned with is likewise fleeting small-scale phenomena/applications. The distinction goes like this: verifying one equation vs. a full-fledged system model; evaluating the efficiency or sustainability of a building in the near-term versus taking into account potential Earthquakes, and extreme weather events in the long-term; innovations in greenhouse horticulture (controlled environment) vs. open-field agriculture (with many variables). It is mainly the latter type of research (which is arguably more apt to shed light on reality) that is incompatible with the prevailing research program structure. As an Ecologist myself, I find it annoying that nearly every single ecological study concludes with a note that “more long-term research is needed”, but the funding agencies have just barely begun taking up this message several decades late.</p><p id="2a3e"><b><i>Extra: </i></b><i>The entire field of psychology is under reproducibility crisis (<a href="https://discovery.dundee.ac.uk/ws/files/7385883/RPP_SCIENCE_2015.pdf">1</a>, <a href="https://journals.sagepub.com/doi/pdf/10.1177/2515245918810225">2</a>, <a href="https://theconversation.com/is-psychology-really-in-crisis-60869">3</a>)</i></p><figure id="136a"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*YTHxuz4ogc15GNRq5V0OWg.png"><figcaption>The distribution of more than one million <i>z</i>-values from Medline (1976–2019). Figure extracted from <a href="https://onlinelibrary.wiley.com/doi/full/10.1111/stan.12241">van Zwet, E. W., & Cator, E. A. (2021)</a>. The significance filter, the winner’s curse, and the need to shrink. Statistica Neerlandica, 75(4), 437–452. (Fair use application)</figcaption></figure><h2 id="da24">Moving forward? Or backward?</h2><p id

Options

="0092">As aforementioned, the current research “progress” runs deeply on snowballing, reputational effects based on trust that keeps marching forward. In other words, we lost a <a href="https://www.nature.com/articles/ejcn201417">critical culture</a> — discussions, and debates that challenge previous ideas, in exchange for more and quicker <a href="https://www.vox.com/science-and-health/22360363/replication-crisis-psychological-science-accelerator">positive results</a>. The new positive results don’t critically amend previous ideas, they instead try overshadowing them by marketing and sheer number.</p><p id="acff">Why has the critical culture been lost? In the current research environment, unless a high-profile researcher (who is miraculously not busy) comes across a sub-optimal or fraudulent paper, any problem not detected in peer review would likely go unchallenged. Inexperienced “small potatoes” are unlikely to conduct an independent verification test, write a rebuttal or complaint even when they strongly suspect a mistake or outright fraudulency. First, it takes time and resources to do so (where would fundings for such “boring” replication tests come from?). Second, journals have a lower tendency of <a href="https://www.youtube.com/watch?v=42QuXLucH3Q">publishing null or negative results</a>. Third, no amount of “cross-checking” work will get you a Ph.D. degree. Lastly, it might potentially stain their career if some of the authors in the disputable paper happen to be high-profile i.e., challenging the relentless forward wheel of the publish-or-perish game is always “costly”. Now that we come back to “cost”, it is easy to see how this is tightly connected with the capitalistic rat race, which punishes non-conformers heavily.</p><p id="3660">The strength of non-conformers comes in number and unity. We need to normalize going backward, backward in a beneficial sense. In other words, we need more robust negative feedback mechanisms in place, be it the re-investigation of “accepted” paradigms, replication study, paper retraction, or facilitated de-growth, or deceleration. If you are a serious thinker, it is not hard to understand that robust negative feedbacks are essential for stable progress, it is the eternal philosophy of all life. There is no reason to believe we can succeed otherwise.</p><p id="5ef9">The modern research structure resembles a pyramid scheme with very unequal power distribution from top to down. When the science produced is good, this model grants benefits to a few big names, but when bad science gets caught, the responsibility is diffused among many. You would be surprised how many <a href="https://www.nature.com/articles/d41586-018-06185-8">co-authors</a> a typical high-ranking researcher has and how big a bad research sub-field can grow (e.g., check out <a href="https://blogs.scientificamerican.com/cross-check/my-problem-with-e2809ctabooe2809d-behavioral-genetics-the-science-stinks/">behavioral genetics</a>). If bad science is done by everyone, it is hard to catch the “culprit”. There is thus little incentive for the Principal Investigator — the one who bargains for funding, to deviate from a routine, convenient path that worked in the past. Breaking the rat race cycle requires non-conformers, but going <b>backward</b> and <b>sideway</b> is easier said than done. The take-home here is that the first step to reduce the cost of changing direction appears to be the disruption of the current incentive structure by trickling the power <b>downward</b>.</p><p id="36e1">To end, I am not telling people to lose trust in science and experts because the Scientific Method is undoubtedly the best out there to reveal objective reality, just that there are problems with its execution and interpretation. As you already saw, this method-execution dissonance has every bit to do with the delusive culture of capitalism that promises never-ending growth and accumulation, a numbered ladder to glory. It is hard to blame the dissidents for their distrust because this is a built-in feature — a requirement rather than a freely adjustable variable.</p><p id="1ab1">Self-perpetuating laissez-faire capitalism (without adequate negative feedback), by operation, must systemically fail to deliver promises. Otherwise, there is no room for new products to keep springing up and profits to accumulate. A question all scientists/truth seekers should ask themselves is then — is “systematically failing” the best way to gain insights into objective truths and reality? After all, the reason “new” sciences have to keep popping up is that previous sciences were not done well (that’s why the faster new sciences are produced, the more critical we need to be). If we instead tame down our pace and ego to build reliable science slowly and critically, it is very hard to conceive why science should “systematically fail” continuously.</p><p id="4b8d"><i>If you like this story, please clap, <a href="https://medium.com/@marmotian/subscribe">subscribe</a> to me, and share it far and wide! Support me <a href="https://marmotian.com/contact/">here</a></i> and <i>join Medium membership <a href="https://medium.com/@marmotian/membership">here</a>.</i></p><p id="9ce5">Read my story probing into the research field of evolution here:</p><div id="e783" class="link-block"> <a href="https://readmedium.com/your-genes-are-not-selfish-part-3-the-political-story-and-neutral-science-10e47f0f3609"> <div> <div> <h2>Your Genes Are Not “Selfish” (Part 3) — The Political Story and Neutral Science</h2> <div><h3>You and I, including “public gurus” may not be impartial and infallible humans. Truth is at the mercy of overpowered…</h3></div> <div><p>medium.com</p></div> </div> <div> <div style="background-image: url(https://miro.readmedium.com/v2/resize:fit:320/1*aK8SdW1BLTLYfWEF2BhdFg.jpeg)"></div> </div> </div> </a> </div></article></body>

The Publish-Or-Perish Game

When science is being dragged into the omnipresent rat race

Let’s publish more and more - all to the garbage bin in the end! (Image from Wikimedia Commons)

“[Insert the latest issue] is a hoax!” — millions of people “Dr. X is a fraud!” — millions of people “Science is dead!” — millions of people

Chances are, you have come across comments like these, more and more frequently so. You’d see them so long as there is a tiny bit of science and tech — on news of COVID (both vaccines and the pandemic itself), monkeypox, cancer, climate change, evolution/paleontology, and even the Moon landing. I am not trying to side with one (anti-science) or the other (scientism), because you can’t, these comments are coming from all angles and contexts, some are more justified and others are simply troll comments from zealots.

However, I am going to tell you one thing very honestly: the robustness of “scientific research” (not the Scientific Method), in general, is on the decline, and this is driven by our entrenchment in capitalism i.e., the eternal rat race to accumulate some numbers, which manifest itself as the “publish-or-perish” game in academia.

Who doesn’t want to publish 70+ papers per year? Who doesn’t want their name to be seen in a headline like: “Dr. X has made the most important discovery about Y of this Century, which will constitute a perfect solution Z”? This is the holy grail of science, right?

Hardly, I would say. This is simply the holy grail of egomaniac “scientists”, not science itself. Science and reality don’t give a damn about personal glory, unfortunately. The ultimate goal of science is to shed light on (and describe) objective reality. Reality is a messy, complex terrain, especially when matters involve the prediction of the future (or far past). True scientific advancement takes the form of long, meandering journeys gathering evidence, branching yet converging over time. “Scientific problems” are also categorically different than “engineering problems” resolvable by known physics, they are complex by nature and cannot be fully solved by one man, one study. Repeated, independent reproduction of multiple lines of evidence is the core of science.

Extra: Prominent cancer researcher — Bharat Aggarwal, has the highest tally of papers being retracted upon his retirement, after being investigated for image and data manipulation.

Of course, there are geniuses like Newton, Darwin, and Einstein who contributed substantially to science single-handedly. But strictly speaking, they too built their acclaimed theories on the shoulder of previous or contemporary scholars. Furthermore, by mathematical necessity, there can’t be many of these paradigm-changing geniuses, if everyone can go straight to become the “star of the show”, science would be in constant turnover and turmoil, nobody could trust a thing at any given time. We need to innovate, update and amend science for sure, but we can only do so reliably at a moderate intensity and frequency. There is only one objective truth in the end.

There is a limit to how many “Einstein” the world can produce. We aren’t going to see many world-beaters or silver bullets. Physics has done a good job sticking with “old” but successful theories like General Relativity and Quantum Mechanics, much effort is done to verify, extend and combine them, not overthrowing or overshadowing them. (Image from Wikimedia Commons Public Domain)

Money makes fame, fame makes money

You now would ask, where do such egomaniac motivations come from? My answer is simple: the widespread normalization of capitalism (profit accumulation) and the commodification of everything (exploiting all possible means of profiting). I dare say that in the times of Newton, and to a lesser extent Darwin and Einstein (their times were influenced by Industrial Revolution and sociopolitics surrounding wars), the development of science was propelled more by a genuine desire to make discoveries and improve our understanding of natural laws. Simply, novel scientific discoveries used not to earn you any good reputation (unless it makes certain merchantable products) and most have been heavily ridiculed by religious powers — there was almost no “scientific corporate ladder” to climb. That’s why you see many scientists/natural philosophers in the old days got their recognition only posthumously.

Nowadays, the titles — scientists, specialists, health experts, etc. merely represent some standardized institutional jobs belonging to the corporate ladder i.e., one has to climb ahead of another in a rat race to get a hold of the position. One key distinction is that the importance of external money is tied directly to securing a scientific job in the first place (not just the continuation of the job) i.e., the ladder is there, but you can’t climb it if you don’t secure research funding first. It’s not hard to imagine then a small percentage of researchers would fake/sensationalize how promising and “novel” their research is, by manipulating words, data, or statistics, or backstab rival ideas and theories by misrepresenting them. Similarly, for tech or application research, it is necessary to create an apparent appeal of their products in the market — science sells. These motivations coupled with the advancement of financial technologies that can move money around quickly have resulted in immense pressure of hype and fraud creeping into every corner of scientific research, arguably much more pervasive than in the old times.

Admittedly, the reliability of scientific knowledge is easily undermined in such a research environment. Scientific research bears a “compounding” reputational effect, in that if one has successfully gotten funding for a particular type of research, they are more likely to keep the money (funding) pumping by continuing to further develop this original narrative (rather than ditching it and changing research direction). Same in paper publication, after having gone through the peer-review process, statements/knowledge embedded in a paper are automatically taken as some kind of serious “truths”, and its dissemination runs almost purely on trust. That is, under publish-or-perish, even a slight misrepresentation in the beginning can snowball into a whole field of problematic research due to the obsession to maintain funding and fame (some countries even have cash rewards for every paper published!). Research success now comes from getting past a one-step initial gatekeeping and subsequent high output numbers, less the actual quality of science.

Extra: Fraudulency in “influential” Alzheimer’s studies has spawned an entire field of questionable research

“Rigged” research structure

Money manipulates. A capitalist-minded funder of course has a tendency to seek those who can write a good story/proposal with the appeal of a “silver bullet”. Frankly, in a dog-eat-dog economy, who would fund a 5-decade research program that does not promise to deliver a clear, attractive goal? Researchers either have to break the program down into small pieces (aka. salami slicing) or else they would have to sensationalize its attractiveness, the PR surrounding nuclear fusion research is a good example in action. There is this ubiquitous mindset that the more and quicker you invest in a particular “attractive” research, the faster the next “breakthrough” will come, irrespective of what’s going on in the rest of the world.

It is no coincidence that there is a glaring pervasiveness of short-term, small sample-size research plaguing all of Science nowadays. Such handy programs are attractive because they are methodologically less standardized (i.e., more easily manipulated via p-hacking to achieve “statistical significance”) and can produce quick deliverables (i.e., the illusion of something being “done”), despite being unable to generate reliable knowledge and insight. Unfortunately, science is never about “attractiveness”, it is about continuous testing and reproducibility, refuting or refining previous theories, however “boring” it may sound. Every researcher probably understood this at some level in the back of their mind, but the drive of capitalism is one hell of a drug that promotes mind-body dissonance.

This is not to say that individual miniature studies are necessarily worthless, but they only work if what you are concerned with is likewise fleeting small-scale phenomena/applications. The distinction goes like this: verifying one equation vs. a full-fledged system model; evaluating the efficiency or sustainability of a building in the near-term versus taking into account potential Earthquakes, and extreme weather events in the long-term; innovations in greenhouse horticulture (controlled environment) vs. open-field agriculture (with many variables). It is mainly the latter type of research (which is arguably more apt to shed light on reality) that is incompatible with the prevailing research program structure. As an Ecologist myself, I find it annoying that nearly every single ecological study concludes with a note that “more long-term research is needed”, but the funding agencies have just barely begun taking up this message several decades late.

Extra: The entire field of psychology is under reproducibility crisis (1, 2, 3)

The distribution of more than one million z-values from Medline (1976–2019). Figure extracted from van Zwet, E. W., & Cator, E. A. (2021). The significance filter, the winner’s curse, and the need to shrink. Statistica Neerlandica, 75(4), 437–452. (Fair use application)

Moving forward? Or backward?

As aforementioned, the current research “progress” runs deeply on snowballing, reputational effects based on trust that keeps marching forward. In other words, we lost a critical culture — discussions, and debates that challenge previous ideas, in exchange for more and quicker positive results. The new positive results don’t critically amend previous ideas, they instead try overshadowing them by marketing and sheer number.

Why has the critical culture been lost? In the current research environment, unless a high-profile researcher (who is miraculously not busy) comes across a sub-optimal or fraudulent paper, any problem not detected in peer review would likely go unchallenged. Inexperienced “small potatoes” are unlikely to conduct an independent verification test, write a rebuttal or complaint even when they strongly suspect a mistake or outright fraudulency. First, it takes time and resources to do so (where would fundings for such “boring” replication tests come from?). Second, journals have a lower tendency of publishing null or negative results. Third, no amount of “cross-checking” work will get you a Ph.D. degree. Lastly, it might potentially stain their career if some of the authors in the disputable paper happen to be high-profile i.e., challenging the relentless forward wheel of the publish-or-perish game is always “costly”. Now that we come back to “cost”, it is easy to see how this is tightly connected with the capitalistic rat race, which punishes non-conformers heavily.

The strength of non-conformers comes in number and unity. We need to normalize going backward, backward in a beneficial sense. In other words, we need more robust negative feedback mechanisms in place, be it the re-investigation of “accepted” paradigms, replication study, paper retraction, or facilitated de-growth, or deceleration. If you are a serious thinker, it is not hard to understand that robust negative feedbacks are essential for stable progress, it is the eternal philosophy of all life. There is no reason to believe we can succeed otherwise.

The modern research structure resembles a pyramid scheme with very unequal power distribution from top to down. When the science produced is good, this model grants benefits to a few big names, but when bad science gets caught, the responsibility is diffused among many. You would be surprised how many co-authors a typical high-ranking researcher has and how big a bad research sub-field can grow (e.g., check out behavioral genetics). If bad science is done by everyone, it is hard to catch the “culprit”. There is thus little incentive for the Principal Investigator — the one who bargains for funding, to deviate from a routine, convenient path that worked in the past. Breaking the rat race cycle requires non-conformers, but going backward and sideway is easier said than done. The take-home here is that the first step to reduce the cost of changing direction appears to be the disruption of the current incentive structure by trickling the power downward.

To end, I am not telling people to lose trust in science and experts because the Scientific Method is undoubtedly the best out there to reveal objective reality, just that there are problems with its execution and interpretation. As you already saw, this method-execution dissonance has every bit to do with the delusive culture of capitalism that promises never-ending growth and accumulation, a numbered ladder to glory. It is hard to blame the dissidents for their distrust because this is a built-in feature — a requirement rather than a freely adjustable variable.

Self-perpetuating laissez-faire capitalism (without adequate negative feedback), by operation, must systemically fail to deliver promises. Otherwise, there is no room for new products to keep springing up and profits to accumulate. A question all scientists/truth seekers should ask themselves is then — is “systematically failing” the best way to gain insights into objective truths and reality? After all, the reason “new” sciences have to keep popping up is that previous sciences were not done well (that’s why the faster new sciences are produced, the more critical we need to be). If we instead tame down our pace and ego to build reliable science slowly and critically, it is very hard to conceive why science should “systematically fail” continuously.

If you like this story, please clap, subscribe to me, and share it far and wide! Support me here and join Medium membership here.

Read my story probing into the research field of evolution here:

Science
Writing
Slowliving
Productivity
Publishing
Recommended from ReadMedium