Are the Models Still Wrong?
Considering a different probability distribution.
In one sense, the Coronavirus models can’t be wrong. The University of Washington’s Institute for Health Metrics and Evaluation (IHME) has varied its mortality estimates since first projecting 84,000 last month. At one point, the estimate rose to 94,000, then it dropped a few times to around 60,000 as of their April 17th estimate. As of this writing (4/21) it now appears to be 66,000. But, as the modelers point out, all of their estimates are within their initial confidence interval of 40,000 to 150,000 deaths, so what’s the big deal?
Suppose I drive from Houston to Dallas and predict that it will take between one and one hundred hours to arrive home. I am almost surely correct, but I’m not giving useful information to my spouse preparing dinner. She wants to know the exact time, and America is focused on the exact numbers.
The IHME models provided critical forecasting of hospital utilization requirements. They also predicted fairly accurate peak usage. However, they may not be as accurate as possible on the back end. In fact, rather than moving downward, ultimate mortality might be a lot higher than 66,000. We have lost 45,000 Americans, including 2,500 today.
[Full disclaimer, I am not a statistician. However, earning a PhD, even in Public Administration, I had to do high level statistics. Also, probability distributions underlay a lot of what quality engineers do.]
The current model
On March 30th, the IHME posted a pre-print paper that outlined its methodology, likely developed in mid-March. The researchers did not have much post-peak data available to plug into the model. Italy, for example, peaked on March 27th.
The model works by using various parameters to estimate the effect of social distancing and modeling the infection rate. From this, a Gaussian curve is fit to the data and carried forward to make predictions. The total area underneath the curve depicts and predicts total deaths.
A Gaussian curve is a the typical bell-shaped curve seen in almost every natural process, regarded as a ‘normal distribution.’ Sometimes skewness distorts the curve in one direction the other. Normal curves run both flat and wide or tall and skinny, expressed in another parameter called kurtosis. Current graphs on the IHME website which reach the predicted number of 65,976 show little or no evidence of skewness.
Whether flat, tall, or skewed to one side or the other, there remains a sinuous shape that slides to zero. The actual data, however, visually shows a little different pattern. Navigating to the IHME website and plugging in Italy, Spain, or New York reveals that while the rise to the peak follows a normal-appearing curve, the decline does not appear to be following the model. There is a distinct on the decline side of the case rates.
For an easier to read example, navigate to Worldometer and check out the graphs for Italy and Spain. The daily new case rate is not following a normal distribution, nor is the daily death rate which lags behind.
The IHME has, to its credit, has started to recognize that the data does not behave quite as originally thought:
Since our last release on April 13, our research team members have adopted a statistical process for identifying which locations have experienced peak daily COVID-19 deaths and how long these peaks last on average…
…Why is this important? The duration of COVID-19 epidemic peaks appears to vary widely across locations (e.g., some experience a sharp peak, while others seem to experience a flatter peak where similar peak daily COVID-19 deaths occur for several days before the epidemic curve declines)…
Misleading data and conclusions
New case rates for COVID-19 do not really reflect new cases. The new case rates reflect the number of people sick enough at the right place and time to get tested. New York’s positive rate in the last week ranged from 25–40% on any given day. Many more people contract the virus than receive tests.
The IHME model suggests that the death rate will be down to less than 10 per day by the end of June for the nation as a whole. The model predicts that New York will be down to zero deaths per day on May 26th. This is absurd.
Consider South Korea. That country sustained 10,683 cases to date. Their infection peak occurred March 3rd. Unlike every other country, their top-line number probably accounts for the vast majority of infections. April 3rd was the last day with over a hundred new cases, and a couple of people are still dying every single day. New York alone had nearly twice as many people with symptoms test positive yesterday as South Korea has had in total. Unlike South Korea, the asymptomatic and lightly symptomatic are not counted. One would expect New York to still see a couple hundred deaths per day one month from now.
Whether Coronavirus takes 60,000 or 80,000 or 100,000 lives, modelers will still argue that it’s in the margin of error. While true, there is a vast difference between taking 4 hours, 8 hours, or 72 hours to drive from Houston to Dallas.
Even more important than the actual number is the expectation that the neat bell curves set. If we only keep up social distancing, cases and deaths will go to zero in another month. This is simply not true.
The mechanics of both infection and mortality distort the shape of the bell curve. The average onset of symptoms is around 5 days, but symptoms could take two weeks or longer. This keeps the number of newly symptomatic cases from dropping sharply.
On the mortality side, typical death, when it occurs, happens at around two weeks. Some deaths happen much quicker, but by definition half of the deaths occur between 0 and 14 days from onset of symptoms. However, there is no upper bound. Putting the worst case together, a victim might become infected and take two weeks to show symptoms, another two weeks until hospitalization, and then linger for a month or more in the ICU.
Alternate distributions
Numerous probability distributions exist. Many of them look similar, but they each serve a specific function. In manufacturing, a ‘Bathtub Curve’ can be derived from the Weibull function to describe reliability of product. Newly manufactured products have defects, which reveal themselves shortly after being used. This leads to early mortality and a high initial failure rate when plotted against time.
After initial problems shake themselves out, products perform over a period of time with relatively few failures along the bottom of the bathtub. As widgets reach the end of their life, the failure rate rises sharply. An example of this is a new building. Some number of light will fail in the first month, a few years will pass without a burnt out bulb, and then lights will seem to fail all at once.
This may also be referred to as a hazard function. The opposite of a hazard function is a survival function. Perhaps what is needed is a properly formulated survival function opposite to the Wiebull function, such as the Inverse Weibull Distribution. Survival functions predict the probability of something, like a patient, lasting a specified period.
As an alternative, it may instead be possible to devise parameters for skewness and kurtosis for the Gaussian distribution. This lopsided bell curve with a long tail into the future could better reflect what the data is starting to clearly indicate.
In either case, the long high tail of cases and mortality from peak new cases and deaths needs to be recognized and accounted for. This large tail on the distribution will mean higher numbers of casualties than the current 66,000, and will mean a much lengthened wave than currently envisioned. The longer it takes to begin managing the public’s expectations, the worse the politics will be.
If you like this article, you may also enjoy:
Brian E. Wish works as a quality engineer in the aerospace industry. He has spent 29 years active and reserve in the US Air Force, where he holds the rank of Colonel. He has a bachelor’s from the US Air Force Academy, a master’s from Bowie State, and a Ph.D. in Public and Urban Administration from UT Arlington. The opinions expressed here are his own.
