Are many youths who survived an attempted suicide living happily ever after?
A peer-reviewed article offers false promises and poor clinical recommendations rooted in suicidology research that has gone bad.
In this installment, I debunk the extraordinary claims of a team of well-published academic psychologists who believe that they can predict which youth who attempt to die by suicide will go on to lead exceptionally happy and productive lives.
We need to be toughminded in interpreting research that is intended by psychologists to inform real-world, life and death decisions. We need to alert consumers if authors are giving foolish advice — and discourage other researchers from falling into the same trap.
The short answer to the question posed in my title is “No, of course not, most youths who make a serious attempt to end their lives cannot expect a dramatic boost in their well-being.” I am rejecting the authors’ claims that life for survivors of suicide is not as pessimistic as other experts and plain common sense tell us. To be persuasive, I will have to unpack some odd things the authors said in a peer-reviewed article and explain where I think they go wrong.
The claims we will discuss are found in a recent peer-reviewed paper in Behavior Therapy:
Tong B, Devendorf A, Panaite V, Miller R, Kashdan TB, Joiner T, Twenge J, Karver M, Janakiraman R, Rottenberg J. Future Well-Being Among United States Youth Who Attempted Suicide and Survived. Behavior Therapy. 2021 Dec 3.
The article is paywalled, but clicking on this link at least provides free access to the abstract and bullet points of the article’s highlights. Here is the start of the abstract:
To what extent does a suicide attempt impair a person’s future well-being? We estimated the prevalence of future well-being (FWB) among suicide attempt survivors using a nationally representative sample of 15,170 youths. Suicide attempt survivors were classified as having high FWB if they reported (a) a suicide attempt at Wave I, (b) no suicidal ideation or attempts over the past year at Wave III (7 years after), and © a well-being profile at or above the top quartile of peers.
The authors believe that they have found a novel way of estimating how many people who attempt suicide will later be doing as well or better than an extreme group of people they have selected who have not attempted suicide. Not only that, the authors believe they can tell us something clinically useful about which factors are associated with survivors doing much better.
I already see glitches in the authors’ logic. Their idea of FWB requires the authors to know ahead of time how many people who identified themselves at one point as survivors of attempted suicide (Wave I) will report doing quite well in a follow-up interview conducted seven years later (Wave III). The authors are not really making a prediction of future well-being only knowing the present and the past, they are counting on a peek at the future. The authors slip into the false assumption that knowing seven years later what will happen to some survivors of attempted suicide can be turned into valuable predictions for other persons who have just survived a suicide attempt and whose future is quite uncertain.
Epidemiologists know that in hindsight we can explain the past from conditions at an earlier or later past for a particular sample, but that does not translate into being able to foresee the future for an entirely new sample. That is in the realm left to soothsayers, fortune tellers, other quacks, and some “experts” who present at suicidology conferences.
Monday morning quarterbacks can always perfectly explain last Sunday’s football score. That does mean we should rely on them for placing our bets about the score of next week’s game.
I am alarmed that the authors exude false confidence in the inaccurate odds that they calculate. I could envision the figure could be misused by some desperate people who are actually considering a suicide attempt. They might decide to go ahead, betting that their life will be improved if they survive.
I read through the full article that I obtained from my university library. My worries increased about authors’ lack of humility and strength of the conviction in their beliefs that they communicated.
The authors started the wrap-up discussion at the end of their article by dismissing what most experts and laypersons believe to be the likely dire consequences of making a serious attempt to take one’s own life. The authors label this accepted view as the “bleak prognostic outlook for people who attempted suicide.”
Like any rational reader, I do not completely put aside what we already know from other sources in reading new things. Science nerds like to remind us to take into account “prior probabilities.”
In this instance, it would take a lot to convince me to ignore what we know are dim prospects that someone surviving a suicide attempt will feel a lot better as a result. We also know that suicides are relatively uncommon among youth and that predicting that a particular youth will attempt and not die by suicide will be impossible. I can only say “Wow! Really?” to these authors thinking that they have a basis for emphasizing possible benefits to someone having attempted to die by suicide.
Many factors determine whether, when, and where a particular person will make a suicide attempt and whether the attempt will end a life. Many of these factors will be unknowable ahead of time or too late to intervene. None of the risk markers we can identify will prove clinically useful in predicting who will be the next youth to die by suicide.
I am also skeptical whether positive factors will add much to traditional negative factors in predicting individual suicides. Neither positive nor negative factors predict very well, and positive factors are mostly turn out to be the flip side (or polar opposite) of negative factors. Experts know something about the role of protective contextual factors like having a spouse in whom to confine or multiple young children living in the home, but no competent clinician would discharge recently admitted self-harming persons from the emergency service solely on that basis of their having these protective factors.
I am even less confident in relying on endorsements on a checklist of subjective emotional feelings, whether positive or negative. Lots of research papers describe other authors’ determined efforts to construct complex scoring keys based on what participants say in response to multiple questions. None of these efforts in the past has proven that this complexity is better than relying on a simple question, which other research has shown is also insufficient. There is no getting around this in predicting attempting or dying from suicide because these events are so much less frequent than the common factors we are using to predict them.
The authors believe that they can overcome these objections because:
*“Our population-based FWB criteria were strict and likely identified individuals who unequivocally met a high threshold of well-being.”
The authors acknowledge that their results require replication. Yet, they shout out to a broad audience who might be impatient about waiting to act on the authors’ astonishing claims until after results are in from future research:
Having credible estimates of FWB after a suicide attempt is important to multiple constituencies, including mental and medical health professionals, as well as individuals who are impacted by suicidal thoughts/behaviors. If these results are corroborated in subsequent work, it would suggest that clinicians should include the prospect of FWB as among the range of possible patient outcomes when discussing prognosis. Indeed, that some previously suicidal individuals achieve FWB may be useful to enhance hope and self-efficacy among individuals who are currently struggling with suicidal thoughts/behaviors, as well as their family members and loved ones. Carefully disseminating the findings about positive trajectories may be useful in addressing the enduring stigma of suicide-related behaviors that burdens suicide attempt survivors (Carpiniello & Pinna, 2017).
The authors call for other researchers to take up the study of FWB to learn:
*Which specific facets/domains of well-being (e.g., self-esteem, family relationship quality) are the most critical for explaining variance in future well-being among suicide attempt survivors.
This is not my first encounter with the thinking of these researchers. I was prepared to be skeptical because I recently conducted a similar “drive-by, cursory review of their work.” I concluded that a key paper that was garnering lots of attention should be retracted because all of its major claims could be shown to be false or exaggerated.
I also disputed some of these authors’ very similar claims that we should not burden depressed persons with pessimism about their prospects for living exceptionally well afterward by bringing up the likelihood of their getting depressed again. The authors believed that negative prognostic statements (“You will likely become depressed again.”) could become a self-fulfilling prophecy.
Clinical depression or more precisely, major depressive disorder, is a recurrent, episodic disorder for which incomplete recovery and relapse/recurrence are highly likely. It might be advisable, especially after a second episode, for depressed persons to set some decision rules about seeking help early rather than waiting until they are in a full-blown episode.
Before reading beyond the abstract of this paper, I could predict that these authors would be immodest in offering an idea that was controversial because it contradicted what we already know. The authors would look for support for their novel idea in survey data that was collected by someone else for a different purpose. They would ignore the limitations of this data and violate a lot of best research practices in choosing what information to extract from the surveys and how to reduce it for analysis.
I could expect that these authors would violate some basic rules for statistical analysis and yet get what most experts considered disappointing results. The experts could confidently say “I told you so” in response to these predictable results. Instead, the authors celebrate their findings and attempt to draw premature attention to them.
The results the authors published here in Behavior Therapy relied on only some of the available data. The authors violated best practices in not publicly declaring research plans ahead of time. It is a common courtesy to ensure there was no peeking at the results before selecting, re-scoring, and analyzing their data as reported in their published paper.
The authors obtained the data for this paper from the publicly available Add Health data set that was collected by other researchers who have widely published many results relevant to their original research questions.
The Wave I Add Health data consisted of 20,745 home interviews. A total of 15,197 Wave I participants completed follow-up interviews seven years later, which became the basis of this study. (Only Waves I and III were included because Wave II did not have relevant questions).
The well-documented untrustworthiness of what is found in psychology journals in the second decade of the 21st century has led to numerous proposed reforms. Among them, researchers should publish their specific plans ahead of conducting a study. There are journals and websites where this can be done, but it was not done for this study.
We cannot tell if the authors’ odd revised scoring was based on peeking at the extensive results that the original Add Health researchers had already reported. The authors might also have finagled their own scoring when they did not at first get the results they wished to obtain. The arbitrariness and even whimsical nature of what they ended up doing encourages skepticism.
The authors were confused about how the nature of the Add Health surveys and how they were limited in what they could say about the lasting effects of a suicide attempt.
Wave I of the survey actually provides only cross-sectional correlational data (“How are you doing right now?”) with retrospective questions about any suicide attempts in the previous year. Causality would be difficult to establish and reverse causality would be plausible.
Participants reporting that they had attempted suicide could have been referring to as long as 364 days ago. Their recollection of what happened could be biased by their current well-being. Alternatively, they might have made a dramatic but non-life-threatening gesture in a situation in which they were lucky enough to get changes in a troubled relationship, such as a spouse ending a trial separation. We cannot tell the difference between very different scenarios in this study.
The authors also ignored survivor bias. Teens who would otherwise be eligible for study but who died by suicide in the past year would of necessity be excluded from analyses. This important bias is akin to the actual research concerning what allowed bombers during World War II to survive being hit by enemy fire. A classic study proved invalid because the only planes available for study were those bombers that returned to base.
Furthermore, having only two waves of interviews with a large gap between them meant the authors could not establish a trajectory. We can’t know what happened in between.
With a 7-year gap in follow-up, the authors are making interpretations of what happens from the start of high school into early adulthood. The Add Health researchers who originally designed the interview study tried to accommodate these profound life changes by asking different questions in Wave III than Wave I.
A study started when teens were in high school will require questions about adulthood for a seven-year follow-up, but only the older students are likely to be in the age range where they are married, starting families, and finishing higher education or starting a career.
The authors had difficulty finding parallels in the questions from Wave III to what was asked in Wave I. A number of the scales that they formed from questions in Wave I had to be shortened for Wave III, introducing considerable imprecision in their results.
The Add Health surveys are a poor choice in terms of what they allow us to say about any attempted suicides.
Worse, being dependent on these data also meant the current authors would not be able to take into account great differences in rates of suicide attempts, the lethality (deadliness of attempts), and profound changes in life circumstances.
There is a problem in the infrequency of teens making serious attempts to end their lives. Teen deaths by suicide are tragic wastes of a long life ahead but are fortunately numerically uncommon. In an earlier article, I showed that if we wanted to reduce the ~ 425 teen girl suicides that occurred in a given recent year, we would have to identify and effectively intervene in a population in that age range of over 20.5 million girls.
There are added complications in studying suicide attempts in this age range, most of which are not lethal. There are marked transitions from early teens until early adulthood in terms of rates, the lethality of methods, and the life circumstances of suicide. Most youths in their early teens do not suffer divorce or personal unemployment or have the protection from suicide provided by an intact marriage. young children in the home, or steady employment in a well-paying job.
The original study was essentially an interviewer-administered self-report checklist study, not a probing personal interview.
The interviewers simply asked broad questions and answers in terms of multiple-choice response options.
The questions that can be used to establish the occurrence of a suicide attempt and its context are quite limited. In an ideal research study, these questions would only be used for screening purposes to prompt interviewers to ask further questions. I have supervised structured interviews by research assistants trained to explain questions to respondents and to probe answers. The added expense is justified by the quality of data that is produced.
In this study, participants were initially asked about suicide attempts at Wave I and seven years later at Wave 2:
During the past 12 months, did you ever seriously think about committing suicide?” If participants answered “yes,” they were queried about suicide attempts: “How many times did you actually attempt suicide?” Participants were classified as surviving a suicide attempt if they reported at least one suicide attempt over the past year at Wave I. Nonsuicidal peers refer to participants who reported no suicidal ideation over the past year at both Waves I and III.
The authors’ ability to characterize the nature and severity of any suicide attempt is limited to one complex screening question about whether medical treatment was sought that required participants to give a simple answer:
Participants who attempted suicide were asked, “Did any suicide attempt result in an injury, poisoning, or overdose that had to be treated by a doctor or nurse?” This item was dichotomous (0 = no, 1 = yes).
The scoring of the FWB measure is bonkers.
I decided not to burden my readers with a full analysis, but I will provide a sample of text and I hope readers will agree that I made the right decision:
The high FWB profile required the participants to both have scored above the 50th percentile on at least 4 out of 5 of the well-being domains, relative to gender-matched sample means from the full Add Health sample at Wave III; and have scored above the 84th percentile (i.e., at least 1 SD above the gender-matched sample means) on at least 3 out of 5 well-being domains. The 4 out of 5 and 3 out of 5 thresholds reflect the level of high well-being met by the top quartile of the peers who reported no suicidal ideation at both waves in the Add Health sample (see Table 3).
There was only an abbreviated assessment of these five domains at Wave III. Life satisfaction was reduced to a single item.
The authors’ unorthodox sampling strategy involved throwing away most interviews and keeping an unrepresentative group of participants who met the criteria for attempting suicide and who had positive well-being scores equivalent to the upper 25% among participants who did not attempt suicide or have suicidal ideation at either time point.
The authors can mostly only cite their own work for justifying this unusual strategy. This is not because no one has ever thought of it, but because most survey researchers know it does not work for multiple reasons. The extreme groups will not generalize very well even to the population from which they are drawn because the groups will differ on other variables of interest in unpredictable ways. There will be lots of noise in the data so getting a clear message about what is going on will be difficult.
Selecting only participants with high well-being is not like selecting only girls or boys. We can be confident that depression is not a determinant of someone being female. In contrast, the measures of wellbeing are moderate to highly correlated with other variables of interest. Unless doctored, the measure of positive well-being assuredly must have a high negative correlation with depressed mood. The authors believe that they are studying the effects of positive well-being, but their measure is hopelessly contaminated with other variables. We cannot be confident that they are only studying the effects of their making bad decisions and would have gotten different results with another arbitrary decision.
The problem gets worse when authors have discarded most participants to obtain a group with exceptional positive well-being. With the full range of data obtained from all participants, the correlation may be significant and substantial, but there is still room for some distinction between positive and negative well-being, with interesting differences between participants. However, choosing only participants who are high in positive well-being is likely stacking the deck to inevitably get a group that is also low in negative well-being because of the messiness/imprecision of the measures.
Here is a useful analogy. If we selected only boys based on having tall fathers, we would distort any estimate of the association of father-son height that would be found in the general population. Boys with exceptionally tall fathers will not be as tall as expected because the exceptional height of their fathers is due to circumstances that are accidental or not shared with their sons. This robust phenomenon is given the name “regression to the mean.”
If this were a workshop for early career researchers, we could spend time discussing lots of other problems that can come with authors having an enthusiastic commitment to flawed ideas who are stuck with data and methods that are much less than ideal for what they intended to study. For our more limited purposes here, however, we can simply note that the authors pinned their hopes on statistical analyses that most Experienced survey researchers, methodologists, and statisticians would never expect this strategy to work.
The authors tried to get around insurmountable problems with what they considered the magic of statistical controls. Don’t be put off or unduly impressed by the technical terms in the passage below, because I can easily show that what the authors tried was incorrect.
A multivariable logistic regression examined predictors of FWB at Wave III, using the variables biological sex, parental level of education, depressive symptoms, frequency of suicide attempt, the severity of suicide attempt (indexed by medical treatment due to attempt), and baseline composite well-being.
“Multivariable logistic regression” means a set of variables was entered into an equation to predict whether participants who reported a suicide attempt and were scored high in the composite FWB were still in the high FWB group seven years later.
The authors tried to manage problems associated with the unavailability of important predictor variables and the crude scoring methods with statistical controls. This was done blindly and without checking whether these control variables were actually related to FWB. For instance, the authors knew that girls attempt suicide more frequently than boys, and that girls’ attempts are less likely to be lethal. They controlled for biological sex without further comment. What advantage did the authors think that would give them over analyses without such control?
The authors did lots of things in this analysis that an epidemiologist would not do. starting with throwing in an arbitrary set of variables into the equation without seeing if they actually mattered for Wave III wellbeing. There is no indication that they did the usual check on how high these predictors were correlated and whether some might be left out because they overlapped too much with another. Or whether there was enough variance in a variable after doing a split between suicide attempt survivors into high and not so high FWB. Regression is not a blunt tool to just dump variables into and confidently assume that what comes out can be trusted.
In this analysis, the authors falsely assumed if they had important, interpretable results if they found any evidence that they could reject the null hypothesis (p <.05) of no prediction of FWB seven years later using these variables. Actually, they needed to meet the much higher bar of there being an association strong enough to work in a new sample for which the future was uncertain.
Almost all of their findings are summarized in the table below. If we take these findings at face value, we would have to say that whether participants were exceptionally high in well-being after attempting suicide by the time of Wave 1 were not more likely to attempt suicide in the period covered by Wave III, seven years later. Overall, whether participants were initially being high in positive well-being at Wave I modestly predicted that they would again be high seven years later. Initial depressive symptoms or number and severity (whether any attempts required medical attention) did not seem to matter.
When all else fails, we can expect a variable’s initial value to predict the value at a later date. We can bet on “Checklist assessment of subjective positive feelings predicts later checklist responses,” and expect to right fairly often, without knowing other details of a study. I read and re-read the authors’ self-congratulatory discussion. I have no idea where they got their enthusiasm or why they called for more of the same kind of research.
We don’t nearly understand enough about what attempting suicide does to people’s future prospects. We need a lot less research, but more careful and better-planned studies. We can expect lots of disappointments and only slow progress. Most “breakthrough findings” will prove false.
Research on suicide is mostly not for the impatient or anyone expecting immediately obtaining exciting findings. If they had asked me, I could have told these authors not to bother with all this effort.
Sign up at [email protected] for news about new stuff — new articles elsewhere, new media, and special events — that will be coming soon from James C. Coyne AKA Coyne of the Realm.
