May 13, 2022 | News | Interview

How to Calculate Excess Mortality to Better Estimate the Impact of the Pandemic

Excess mortality is an often-used measure to estimate the mortality burden due to the Covid-19 pandemic. Marília Nepomuceno, Dmitri Jdanov and colleagues looked at factors that affect this measure. In this interview they explain why excess mortality rates vary substantially when changing data and methods and recommend ways to better calculate excess mortality.

Dr. Nepomuceno, Dr. Jdanov, in your recent paper you analyzed some factors that impact excess mortality estimates. What are these?

Marília Nepomuceno (MN): We investigated the impact of four factors on excess mortality estimates:

the mortality index
the method
the number of previous years included in the baseline
the time unit of the death series

All these factors matter, although at different magnitudes.

As you said, all factors matter. How does the proportion of older people in a population affect excess mortality estimates during the COVID-19 pandemic, in particular?

Dmitri Jdanov (DJ): Differences between population age structures are important when comparing populations' mortality levels or estimating excess mortality in a given population. A higher proportion of older adults combined with the steep age gradient in COVID-19 mortality resulted in higher excess mortality rates in older populations than in younger populations during the pandemic. Italy is a good example of this phenomenon because it has one of the oldest populations in the world. The Italian position in country rankings depends on the mortality index used to take the age structure of the population into account.

You also had a look at the reference period, which is the number of previous years used to define the expected level of mortality, the so-called baseline. Why did your findings indicate that for most countries, longer reference periods resulted in lower excess mortality?

DJ: The baseline is a reference level of mortality that would have occurred in the absence of the pandemic. It reflects as accurately as possible the past mortality experience over the reference period. Many of the studies on excess mortality due to the COVID-19 pandemic have arbitrarily chosen the reference period from 2015 to 2019. In our paper, we show that the estimates of pandemic losses are sensitive to the choice of the number of years included in the reference period. For instance, for countries that experienced steeper declines in mortality in the recent years, the expected level of mortality for 2020 based on the reference period 2017 to 2019 will be lower than that based on the longer period. So, if expected mortality is low in a short period, excess mortality will be higher than compared to a long period with high mortality.

What steps are needed to make weekly and monthly death series comparable to achieve similar excess mortality estimates?

MN: This was our initial question that started the project. Weekly mortality statistics are very limited while the monthly death counts are part of the standard statistical reports. As we expected, the data time unit of the death series was the factor associated with the smallest variations in annual excess mortality estimates. Nevertheless, using different time units might lead to incorrect results for a short period like months because of the shift in the reference period. For example, the period that refers to the first four weeks of a year might start between 29 December and 3 January while the first month always starts on January, 1st. Thus, the time frame used to estimate expected mortality will differ.

Your findings also suggest that the method used to estimate the baseline is an important source of variation in excess mortality.

DJ: Absolutely. Using the appropriate method to estimate the baseline is crucial to achieve robust excess mortality estimates. The methods that have been used most frequently during the COVID-19 pandemic are simple (weekly-)specific averages and regression models. In our study, we use four mainstream methods:

the specific average
specific average with the trend
harmonic with the trend
specific trend

This is not an exhaustive list, there are tens of methods in the literature, but it is enough to show the sensitivity of excess mortality estimates. The methods that considered linear trends are more accurate and less sensitive to the choice of the mortality indicator and provide similar excess mortality rates for both the crude death rate and the age-standardized death rate. The magnitude of the variation in excess mortality rates due to the choice of a method that did or did not account for trends changed for each country, ranging from about 130 deaths per 100,000 persons in Lithuania to 26 deaths per 100,000 persons in New Zealand. In addition, the choice of the method matters to understand variations in excess mortality, as we found that the magnitude of these variations was country-specific and depended on the selection of the mortality index.

How big is the variation of excess mortality for the 26 countries you analyzed, depending on the factors used to calculate it?

DJ: We should distinguish two possibilities: absolute changes in excess mortality for a country and relative changes in country rankings. We investigated the sensitivity of excess mortality estimates in 2020 to the choice of the four factors in the following 26 countries: Austria, Belgium, Denmark, England and Wales, Estonia, Finland, France, Hungary, Israel, Italy, Latvia, Lithuania, the Netherlands, New Zealand, Norway, Poland, Portugal, Republic of Korea, Scotland, Slovenia, Slovakia, Spain, Sweden, Switzerland, Taiwan, and the United States.

...and what did you find?

MN: We show that excess mortality rates varied substantially when these factors changed, and the magnitude of these variations changed markedly within countries, which resulted in changes in the country rankings. For example, the magnitude, that means the absolute value of the differences in excess mortality rates changes substantially within countries when the reference period is changed: the point estimate ranges from 0.1 to 55 deaths per 100,000 persons. These changes are not consistent across countries and, as a result, cross-country comparisons based on different methods are significantly different.

In general, what do you recommend to calculate the most accurate excess mortality possible?

MN: The first recommendation is related to the mortality index: When comparing populations, differences in their age compositions should be considered; thus, age-standardized rates are recommended. However, it is also important to highlight that age-standardized rates are hypothetical rates that express what would have been observed in a population if it had the same age composition as the standard population. By contrast, crude death rate express real-life mortality and population losses; as such, they are not misleading when populations are not compared. Thus, we recommend using age-standardized indicators as an additional measure in all cases when crude rates are used.

Are there other recommendations?

MN: Yes, it regards the methods for estimating baseline mortality. These methods should consider mortality trends over time. Another suggestion is for the reference period. The number of years included in the baseline period should be large enough to identify a stable and clear mortality trend. Therefore, the best length for the reference period is country- and model-specific. Countries have been experiencing different changes in mortality over time, which should be considered when estimating excess mortality. Finally, the choice of the model should be task-specific and consider this variation.

Background Info

What is excess mortality?

The excess is the difference between mortality, usually from all causes, that is observed and mortality that would be expected, the baseline mortality. In the context of the COVID-19 pandemic, excess mortality can be estimated by measuring the difference between mortality from all causes that is observed during the pandemic and mortality from all causes that would be expected if the pandemic had not occurred, the baseline mortality. The tricky task is to define the expected mortality. There are several approaches how to define the baseline mortality. However, several crucial aspects should be considered in all cases: mortality trends during past years, changes in population structure, and the lowest possible mortality level in the population. In the STMF visualization toolkit at the Human Mortality Database one can play around with different inputs and methods to estimate excess mortality.

Why is excess mortality a valuable metric to be used during the COVID-19 pandemic?

Excess mortality is one of the most reliable approaches to measure the impact of the COVID-19 pandemic. The main reason is that it considers deaths from all causes. Thus, it is independent of the COVID-19 testing capacity, the definition of COVID-19 deaths, and the misclassification of COVID-19 deaths on death certificates. Moreover, the metric includes losses that are both, directly and indirectly, attributable to the pandemic. As a result, estimating excess mortality has been considered the best approach for assessing and comparing the overall mortality burden due to COVID-19.

For the case of Germany, what would you recommend to calculate the most accurate excess mortality possible?

The answer depends on the research question. Nevertheless, it is clear that the week-specific average based on the 2015-19 reference period is significantly biased for Germany. First, it doesn’t take into account the observed mortality trends and population aging (fortunately, they partly compensate each other). Most importantly is that Germany was affected by three severe flu epidemics in 2015, 2017, and 2018. Including these epidemic waves in calculation leads to a significant underestimation of pandemic losses.

Original Publication

Nepomuceno, M.R., Klimkin, I., Jdanov, D.A., Alustiza-Galarza, A., Shkolnikov, V.M.: Sensitivity analysis of excess mortality due to the COVID-19 pandemic. Population and Development Review (2022). DOI: 10.1111/padr.12475