MPIDR Working Paper

The limits of predicting individual-level longevity

Badolato, L., Decter-Frain, A. G., Irons, N., Miranda, M. L., Walk, E., Zhalieva, E., Alexander, M., Basellini, U., Zagheni, E.

MPIDR Working Paper WP-2023-008, 31 pages.
Rostock, Max Planck Institute for Demographic Research (February 2023)
Revised January 2024 (Former title: Predicting individual-level longevity with statistical and machine learning methods)

DOI:10.4054/MPIDR-WP-2023-008

Publication as PDF

Open Access

Reproducible

Abstract

Individual-level mortality prediction is a fundamental challenge with implications for life planning, social policies and public spending. We model and predict individual-level lifespan using 12 traditional and state-of-the-art models and over 150 predictors derived from the U.S. Health and Retirement Study. Machine learning and statistical models report comparable accuracy and relatively high discriminative performance, but fail to account for most lifespan heterogeneity at the individual level. We observe consistent inequalities in mortality predictability and risk discrimination, with lower accuracy for men, non-Hispanic Blacks, and low-educated individuals. Additionally, people in these groups show lower accuracy in their subjective predictions of their own lifespan. Finally, top features across groups are similar, with variables related to habits, health history, and finances being relevant predictors. We conclude by highlighting the limits of predicting mortality from representative surveys and the inequalities across social groups, providing baselines and guidance for future research and public policies.