Roughly half of Americans are pretty freaked out about COVID-19. People who went to college, make a lot of money, are old, or are white, are especially likely to be freaked out.
On March 3rd, the World Health Organization declared that 3.4% of people who had contracted COVID-19 had died (WHO, 2020). Since then, we’ve seen that some nations are reporting fatality rates 30 times higher than others. So there is a great deal of variance between estimates.
As Oke et al. note, early estimates of a disease’s fatality rate can be biased upwards by the preferential testing of those who have developed symptoms and are in need of medical help. On the other hand, within the set of people who currently have COVID-19 there is a subset that will die from the disease who have not yet. There is also some number of individuals who have already died from the disease but whose recorded cause of death is not COVID-19. The more such people there are the more the fatality rate will be under-estimated.
Importantly, these sampling biases continue to exert their effect regardless of how many people are tested, and so we cannot avoid them by relying on nations like China or South Korea who have tested a large number of people.
Oke et al give three reasons for thinking the true fatality rate may be quite low. First, the reported fatality rate is falling dramatically with time in China.
Secondly, a decade ago swine flue was estimated to have a fatality rate between .1% and 5.1% and ended up having a fatality rate of .02%. Thus, in 2009 the true rate was one-fifth the size of the lowest early estimate.
Thirdly, the fatality rate on the Diamond Princess, a cruise ship that experienced a breakout, has been recorded as .85%, or 45% as large as the general estimate coming from the USA. Now, it may be that some on the ship have simply yet to die of the disease, but this is unlikely to be true for a great number of people, and the population on cruise ships is probably older than average meaning the fatality rate on such ships is probably greater than it will be for the general population.
For these reasons, Oke et al took the lowest national estimate, Germany’s estimate of .25% and cut it in half based on the notion that roughly half of cases are asymptomatic and untested, and produced an estimate of the fatality rate of .125% for the general population and 1% for people over the age of 70. Notably, this would make the mortality rate of COVID-19 similar to that of the seasonal flu.
(Note: I don’t endorse any of the figures given for COVID-19 in this chart. The same biases involved in estimating the fatality rate also bias estimates of transmission and hospitalization rates).
Some have objected to this estimate on the basis that it cannot explain the fatality rate seen in Italy. Importantly, Italy stands out not only for having a high estimate of the fatality rate, but also for having significantly more deaths than other nations even after ruling out the notion that the disease has had more time to spread in Italy as an explanation of its higher death rate.
Italy is exceptional only with respect to its death rate. It is normal with respect to the speed with which the disease is spreading.
It thus seems clear that Italy is not a solid basis upon which to derive a general estimate of the disease’s fatality rate, either because Italy is mismeasuring its own fatality rate due to the biases mentioned above, or because the fatality rate really is higher than average in Italy, or both.
Oke et al. (2020) give the following explanation for Italy’s fatality rate: “the age structure of the Italian population (2nd oldest population in the world); highest rates of antibiotic resistance deaths in Europe which might contribute to increased pneumonia deaths (Italy tops the EU for antibiotic-resistance deaths with nearly 1/3rd of the deaths in the EU). Smoking also seems to be a factor associated with poor survival – in Italy, 24% smoke, 28% men. In the UK, for instance, 15% are current smokers.”
A more powerful objection to Oke et al’s estimate is as follows: Zhaung et al. (2020) estimate that the false positive rate for COVID-19 tests in China is probably greater than 47%. If this figure is similar in other nations, this will cause us to significantly under-estimate COVID-19’s fatality rate. (Though other biases may be present such that, even with a high false-positive rate, we are still on net over-estimating the fatality rate.)
Moreover, while it is true that we initially over-estimated the fatality rate of swine flue by a factor of nearly ten, the opposite was true of the SARS outbreak of 2003 which was initially estimated to have a fatality rate of roughly 4% but which was eventually shown to have a mortality rate of roughly 10% (Ross, 2003; Mahase, 2020). Similarly, the fatality rate estimates for the Ebola outbreak in 2013 had to be revised from 49% to 70% (Mackay et al., 2014).
So it seems that epidemiologists don’t have a pattern of reliably over-estimating fatality rates. Rather, they just have a pattern of reliably being very wrong when it comes to early estimates of fatality rates for new disease outbreaks. Unfortunately, the direction of their wrongness seems random.
Still, the fatality rate estimates are falling in China, the Princess Diamond data implies that we are generally over-estimating the fatality rate, and it seems likely to me that we are missing more cases of infection than we are missing deaths. Given this, if I had to bet I would say that we are significantly over-estimating the fatality rate of COVID-19 and that it is probably significantly below 1%, though there is no rigorous way of determining the exact degree of that overestimation.
For many of the same reasons laid out here, I’d say the same thing about hospitalization and transmission rates.
But the most important take away, in my opinion, is that we shouldn’t be especially confident in any estimate we think is reasonable, nor should we be confident in what the experts are telling us given their track record of being reliably wrong in this exact sort of situation. Whatever our belief, in this situation we should hold it with a great deal of uncertainty.
Also published at Ideas and Data
Become a Patron!
On March 3rd, the World Health Organization declared that 3.4% of people who had contracted COVID-19 had died (WHO, 2020). Since then, we’ve seen that some nations are reporting fatality rates 30 times higher than others. So there is a great deal of variance between estimates.
As Oke et al. note, early estimates of a disease’s fatality rate can be biased upwards by the preferential testing of those who have developed symptoms and are in need of medical help. On the other hand, within the set of people who currently have COVID-19 there is a subset that will die from the disease who have not yet. There is also some number of individuals who have already died from the disease but whose recorded cause of death is not COVID-19. The more such people there are the more the fatality rate will be under-estimated.
 |
| Oke et al. (2020) |
Oke et al give three reasons for thinking the true fatality rate may be quite low. First, the reported fatality rate is falling dramatically with time in China.
 |
| WHO (2020) |
Thirdly, the fatality rate on the Diamond Princess, a cruise ship that experienced a breakout, has been recorded as .85%, or 45% as large as the general estimate coming from the USA. Now, it may be that some on the ship have simply yet to die of the disease, but this is unlikely to be true for a great number of people, and the population on cruise ships is probably older than average meaning the fatality rate on such ships is probably greater than it will be for the general population.
For these reasons, Oke et al took the lowest national estimate, Germany’s estimate of .25% and cut it in half based on the notion that roughly half of cases are asymptomatic and untested, and produced an estimate of the fatality rate of .125% for the general population and 1% for people over the age of 70. Notably, this would make the mortality rate of COVID-19 similar to that of the seasonal flu.
 |
| Resnick et al. (2020) |
Some have objected to this estimate on the basis that it cannot explain the fatality rate seen in Italy. Importantly, Italy stands out not only for having a high estimate of the fatality rate, but also for having significantly more deaths than other nations even after ruling out the notion that the disease has had more time to spread in Italy as an explanation of its higher death rate.
 |
| Bernard et al. (2020) |
 |
| Bernard et al. (2020) |
Oke et al. (2020) give the following explanation for Italy’s fatality rate: “the age structure of the Italian population (2nd oldest population in the world); highest rates of antibiotic resistance deaths in Europe which might contribute to increased pneumonia deaths (Italy tops the EU for antibiotic-resistance deaths with nearly 1/3rd of the deaths in the EU). Smoking also seems to be a factor associated with poor survival – in Italy, 24% smoke, 28% men. In the UK, for instance, 15% are current smokers.”
A more powerful objection to Oke et al’s estimate is as follows: Zhaung et al. (2020) estimate that the false positive rate for COVID-19 tests in China is probably greater than 47%. If this figure is similar in other nations, this will cause us to significantly under-estimate COVID-19’s fatality rate. (Though other biases may be present such that, even with a high false-positive rate, we are still on net over-estimating the fatality rate.)
Moreover, while it is true that we initially over-estimated the fatality rate of swine flue by a factor of nearly ten, the opposite was true of the SARS outbreak of 2003 which was initially estimated to have a fatality rate of roughly 4% but which was eventually shown to have a mortality rate of roughly 10% (Ross, 2003; Mahase, 2020). Similarly, the fatality rate estimates for the Ebola outbreak in 2013 had to be revised from 49% to 70% (Mackay et al., 2014).
So it seems that epidemiologists don’t have a pattern of reliably over-estimating fatality rates. Rather, they just have a pattern of reliably being very wrong when it comes to early estimates of fatality rates for new disease outbreaks. Unfortunately, the direction of their wrongness seems random.
Still, the fatality rate estimates are falling in China, the Princess Diamond data implies that we are generally over-estimating the fatality rate, and it seems likely to me that we are missing more cases of infection than we are missing deaths. Given this, if I had to bet I would say that we are significantly over-estimating the fatality rate of COVID-19 and that it is probably significantly below 1%, though there is no rigorous way of determining the exact degree of that overestimation.
For many of the same reasons laid out here, I’d say the same thing about hospitalization and transmission rates.
But the most important take away, in my opinion, is that we shouldn’t be especially confident in any estimate we think is reasonable, nor should we be confident in what the experts are telling us given their track record of being reliably wrong in this exact sort of situation. Whatever our belief, in this situation we should hold it with a great deal of uncertainty.
Also published at Ideas and Data
Become a Patron!
