When accurate prediction models yield harmful self-fulfilling prophecies

arXiv (Cornell University)

Abstract

Prediction models are popular in medical research and practice. By predicting an outcome of interest for specific patients, these models may help inform difficult treatment decisions, and are often hailed as the poster children for personalized, data-driven healthcare. We show however, that using prediction models for decision making can lead to harmful decisions, even when the predictions exhibit good discrimination after deployment. These models are harmful self-fulfilling prophecies: their deployment harms a group of patients but the worse outcome of these patients does not invalidate the predictive power of the model. Our main result is a formal characterization of a set of such prediction models. Next we show that models that are well calibrated before and after deployment are useless for decision making as they made no change in the data distribution. These results point to the need to revise standard practices for validation, deployment and evaluation of prediction models that are used in medical decisions.

Authors 4

1. Wouter A. C. van Amsterdam

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Department of Data Science and Biostatistics Julius Center of Health Sciences and Primary Care University Medical Center Utrecht , Utrecht , the Netherlands

   University of Utrecht , Utrecht , the Netherlands Heidelberglaan 100 , 3584 CX Utrecht , the Netherlands

2. Nan van Geloven Aachen Department of Biomedical Data Sciences – Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Biomedical Data Sciences Leiden University Medical Center , Leiden , the Netherlands

3. Jesse H. Krijthe

   Delft University of Technology

   Affiliation as printed

   Pattern Recognition & Bioinformatics Delft University of Technology , Delft , the Netherlands

4. Giovanni Cinà