How You Can Avoid the Common Pitfalls of Survivorship Bias

A Rebuttal Against Self-Help Gurus, Billionaire’s Advice and Pseudoscientific Wellness Practitioners

In 1987, researchers noted a curious trend in the vet’s clinic. Cats that fell less than six stories arrived at the vet with fewer injuries than cats that fell from greater heights. The authors of the study themselves proposed that cats reached a maximal speed, called terminal velocity. Once the falling cat hit this speed, it relaxed its muscles and outstretched its legs to minimize injury.

While the argument that cats better ready themselves for a landing from more than six stories sounds intriguing, it doesn’t hold weight. There is a much simpler conclusion that does explain this phenomenon! Fewer cats survived the higher fall. The vet’s office only received the miraculous few cases of the survivors, skewing and biasing this study.

Let’s posit that 1000 cats jumped from above six stories, while 1000 jumped from less than six stories. In our first group of daredevil cats, we may have only ten miraculous survivors, that came away relatively harmed. After these ten cats visited the vet, perhaps only three were seriously injured. We would say that 30% of these cats suffered injuries from their fall.

Meanwhile, of the 1000 cats that jumped from six stories, all of them survived, half were seriously injured. All of them went to the vet, leading someone to conclude that 50% of these cats sustained injuries. Is this a fair, unbiased comparison?

We see many similar forms of this statistical error in our daily lives. The well-recognized phenomenon is called survivorship bias. This bias obscures the likely outcomes of failure with a misplaced sense of optimism. At first glance, this bias doesn’t sound too deadly. However, a plethora of examples shows otherwise.

Another study showed that a car crash survivor presented with a higher chance of having cancer. Survivors of an automobile accident often undergo various screening procedures to detect injuries. A whole-body medical scan would, therefore, find any cancerous growths! In contrast, usually, individuals arriving at the physician’s office do not undergo immediate and extensive screening. Again, the importance of cautiously interpreting data is crucial to understanding this association.

Self-Help and Money-Making Gurus

The quickest detour to success is always a few quotes, an article in the form of a list or an expensive seminar! While following advice from rich, successful people is appealing, it is not a fantastic detour to success. While tens of millions of people may follow this advice, almost all of them will fail. Who is then to say that the few founders who do follow the advice directly attribute their success to this advice from the wealthy? Founding a company or making successful investments relies on a lot of luck.

According to an estimate from mathematician Samuel Arbesman, only 0.0068% of the population is famous. To arrive at this value, Arbesman looked at the number of notable individuals on Wikipedia. Have you heard of all 600 000 notable people in Wikipedia? I’d imagine there are even fewer individuals that become household names. If all 600 000 of these notable people were household names, we would still be drawing advice from a minuscule sample.

We never hear from any of the millions of individuals that attempt and fail at starting their own companies. The amount of hard-working individuals that do not become successful millionaires is much higher than the amount who succeeds. The issue of survivorship bias is well-noted in the behavioural economics literature. We only hear about successful financiers and investors, assuming that their current success predicts future success.

Therefore, when heeding the advice of millionaires and self-help gurus, it’s best to keep this in mind. Even with many successful testimonials, there are many more real stories of individuals who received no benefit from self-help regimes.

We attribute the wealth and success of these figures to their specific advice and wisdom. We take this small sample of individuals while conflating their advice for a roadmap to success. At best, advice from self-help gurus or the rich and famous might cost you some money. Meanwhile, bad advice could tank your business or venture.

From Self-Help to Dangerous Health Advice

Ever hear someone tell you, “Smoking isn’t so bad, my dad chain-smoked every day and lived to 100!”

While there is a strong psychological appeal to justify our bad habits, we must remember that millions of people who smoke do not reach the ripe old age of 100. However, I cannot fault the centenarians. After all, if you live to 120, the temptation to provide strange advice would be overwhelming.

As a thought experiment, suppose I lived to 120 and happened to rub my hands with gravy every morning. Does this become the cause of my longevity? Of course not! There are more insidious cases of survivorship bias disguised as medical advice:

“I didn’t have seatbelts growing up and I’m still alive!”

“I wasn’t vaccinated but I turned out perfectly healthy!”

Survivorship bias persists by providing a positive, optimistic outlet that appears to have sound logic behind it. Several wellness practitioners promise you their method will improve your mental or physical health! However, presenting one positive case while ignoring all of the negative outcomes associated with the advice is misleading. Seatbelts save lives! Meanwhile, vaccines are one of the most cost-effective, life-saving technologies ever invented!

When any such medical advice is tested using a randomized-clinical trial, it fails. Once you begin considering the full scope of evidence, rather than a few positive cases, it is evident that much of this advice is erroneous and potentially dangerous. If you’re interested in understanding how studies determine the efficacy of the treatment without falling prey to bias, read my previous piece here.

Avoiding Survivorship Bias

The best way to avoid these biases from seeping into your life is to think about the origin and scope of your data carefully. Are you receiving advice from a very small sample group like the rich and famous? Are you perhaps receiving a sample of one, where only this one individual experience is provided as evidence? Are the risks of failure or dangerous outcomes included in the advice?

We must stay vigilant and think critically about the claims we hear to determine whether we are experiencing a form of survivorship bias!