Measuring the true toll of the pandemic
- In 1918, there was an anti-mask league in San Francisco
- Vitamin D Supplementation lowers Risk for Respiratory Tract Infections
- What are coronavirus antibody tests?
One of the grim ironies of the covid-19 pandemic is that the world has shown an unprecedented interest in data — and yet there has never been so much uncertainty around official statistics. Facebook and Twitter have become catalogues of terrifying charts, as people around the globe have tried to work out how quickly the SARS-CoV-2 virus is spreading. Meanwhile, academics and journalists have come up with lots of inventive ways to quantify the impact of the pandemic. The Economist has, for example, published analyses of tourist flows, footfall in major global cities and internet usage.
Yet we still don’t know the numbers that matter most. Because countries have different testing capabilities, and many carriers show no symptoms, we don’t know how many people the virus has infected. More worryingly, we don’t even know exactly how many people it has killed. As I reported in early April, the official covid-19 tolls in many countries exclude victims who were not tested before passing away.
It often takes several days to establish and report the cause of death, creating a lag in the data. And if hospitals are overwhelmed, people with other conditions might not get the intensive care they need to survive. As a result, the numbers of deceased that get read out each night on the news are smaller than the true number of fatalities that the virus has already caused.
During medical crises such as this one, a better way to measure mortality is to look at a region’s “excess deaths”: the number of people who have died of all causes, compared with the average for the same period in previous years. I have been using two approaches to do this. The first is to take publicly available data about mortality, produced regularly by a handful of statistical bureaus, and estimate how many more people have died than usual in a selection of places.
The second is to use figures from EuroMOMO, a network of epidemiologists in 24 European countries, which covers 350m people in its weekly reports. As the pandemic continues, I will update all of these datasets on our excess-mortality tracker, which is available free to all readers.
How I calculated The Economist’s estimates
One of the biggest challenges when it comes to measuring the spread of the virus is that every country collects its data differently. This is particularly true for the official covid-19 figures. For example, France has conducted a considerable amount of testing in care-homes, and includes fatalities from such institutions in its daily tolls, whereas almost all of the Netherlands’ reported deaths have occurred in hospitals.
Yet the data about deaths from all causes, which might seem more straightforward to collect, also vary in quality and frequency across countries. Spain’s Carlos III Institute, its national epidemiology centre, is publishing daily figures for each region — but these have a large lag, and are increased retrospectively when newer data come in (see chart below).
By contrast, Britain’s Office for National Statistics produces a weekly total of the number of death certificates registered in its system, which are not revised in future. So the comparisons between countries are not entirely apples-to-apples. We have also had to use a simple method for our baseline of “expected deaths”: an average for the same week in previous years (ideally the past five years, but sometimes two or three, depending on availability).
Moreover, some countries have reported only partial mortality data, which cover a non-random selection of regions or victims. In Istanbul and Jakarta, the two non-Western places currently in our tracker, I am relying on tallies of the number of people who have been buried, as reported by local departments. These figures will not pick up everybody who has died, but they are still much higher than the official covid-19 tolls.
Turkey does not even give regular breakdowns of covid-19 deaths by region, so we had to estimate that Istanbul had half of the national share, based on a report from April 1st. Given how slowly many developing countries publish their all-cause mortality data, I will probably have to use a similar approach to find numbers in South America, Africa, Asia and the Middle East.
Even some rich nations do not have complete data. Italy’s National Institute of Statistics (ISTAT) has released figures on all deaths for just 1,700 of the country’s 7,900 municipalities (or comuni) — only the ones that have both shown at least a 20% increase this year and claim to have accurate records.
This makes it impossible to come up with a national tally of excess deaths in the European country that was hit hardest. However, because these comuni cover 7.2m of the 10m people in Lombardy, the worst-affected Italian region, I was able to produce an estimate for that area. I found a snapshot map of official covid-19 deaths for those comuni at the end of March, and used that to predict what share of Lombardy’s deaths these 7.2m people suffered each week.
All of this data-cleaning is fiddly and time-consuming, especially because the underlying spreadsheets are in different formats and languages. Whenever new data are published, I check them carefully, in case the definitions have changed. New York’s official death toll, for example, has started to include people who “probably” died of covid-19, according to the symptoms on their death certificates.
To make this process more organised, I wrote a script in R, the programming language used most often by data journalists at The Economist, to do the cleaning for me. It is about 1,000 lines long, and spits out CSV files in the same format for each country. Those go into a Github repository, which Martín González, one of our brilliant interactive journalists, uses to make the dynamic charts for our tracker. We hope that, at some point, once we have made the code presentable, to make this resource publicly available — keep checking our GitHub.
How EuroMOMO comes up with its estimates
Although I have been able to find mortality numbers for many European countries, not all of them make these data publicly available in a timely fashion. Germany, for example, has only released nationwide figures up to March 13th (at the time of writing), since like Italy it struggles to coordinate lots of regional offices.
To get around this, I’ve collaborated with EuroMOMO’s epidemiologists. They collect mortality data on a weekly basis across Europe, which we will include on our tracker. And while they don’t break down the absolute numbers of fatalities by country, they do provide a trend chart for each member state on their website.
In this respect, EuroMOMO is a unique resource. No other continent has such a coordinated system. The United States has a “Pneumonia and Influenza Mortality Surveillance” tool, which publishes weekly total deaths for each state. But states take months to update their figures, which is why we have only included the city of New York in our tracker so far.
While working with EuroMOMO, I spoke to Lasse Skafte Vestergaard, to find out more about how the data are collected and what the limitations might be. Lasse coordinates the network, and helps put together the bulletin that the group publishes each Thursday. He is also an epidemiologist at Denmark’s national institute for infectious disease, and has specialised in influenza, malaria and parasitology.
JT: Lasse, your organisation is the only one that tracks excess mortality across an entire continent. How did this come about?
LSV: EuroMOMO’s network was set up in response to the global influenza pandemic in 2009, to ensure early detection of future public health events. Currently we have 28 partner institutions across 24 countries and regions. We also have representatives from the European Centre for Disease Prevention and Control (ECDC) and the World Health Organization (WHO) Regional Office for Europe, who both support the network activities and our hub financially, participate in technical discussions, and help our active strategy to expand geographical coverage, especially in Eastern Europe.
For example, Ukraine has recently started to provide monthly data to the hub. One of the great strengths of our approach has been its consistency and close collaboration between participating countries, running week after week, year after year. Putting together the national data from all of them adds power to detect smaller changes than what individual countries may immediately see.
JT: How comprehensive are your numbers?
LSV: Our data come from national or regional statistical offices, and are typically based on information on deaths recorded in civil registers. Overall, our sample covers a substantial proportion of the European population, with more than 350 million people. However, the fraction covered varies quite a lot between countries, from 95–100% in many to only 15–20% in others. This all depends on local administrative systems and political decisions.
JT: Are there any limitations that people interpreting the numbers should be aware of?
First of all, we are providing mortality estimates, based on certain statistical models and assumptions, not the exact numbers of deaths. There will always be some level of uncertainty around them. Secondly, the coverage in some countries is only for a few selected regions or cities, for example in Germany and Italy.
Since we are looking at national-level estimates, the observed figures may not adequately detect much higher mortality levels within smaller parts of a country. Finally, even the best quality reporting is not fully complete, and there are always a few days or weeks of delay. We apply an adjustment function in the statistical algorithms, but still, the estimates provided for the most recent weeks must be read with caution.
JT: Some readers might look at EuroMOMO’s charts and think that the death toll from covid-19 is no worse than from some of the recent flu seasons. How would you respond?
LSV: Given the dramatic mortality signals we are already seeing, especially in some parts of Europe, we can conclude that this pandemic is not just another flu — not at all. It is striking to see large variation, from extremely high excesses in one country to normal levels in neighbouring ones. We will later have to evaluate the effect of the various societal preventive measures, initiated with different speeds and levels of coverage. Then we can conclude what should be recommended in terms of preparedness for future pandemics.
We don’t yet know how long we will keep updating The Economist’s tracker. If the pandemic surges and then abates over several months, as countries go into and out of periods of lockdown, then excess mortality figures will be an important measure of how well social-distancing policies are working, and when they need to be enforced.
I will keep digging away at official reports and spreadsheets in various languages, and expanding our ever-more tangled R script. If any readers spot data that we could use, or corrections that we need to make to the existing numbers, those would be very helpful — and I would be happy to hear about them at
Source: Medium economist
More News: Coronavirus may last 2 years and second wave could be worse , study warns