Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models
Annals of Internal Medicine, vol. 176, pp. 105–114
Abstract
Risk prediction models need thorough validation to assess their performance. Validation of models for survival outcomes poses challenges due to the censoring of observations and the varying time horizon at which predictions can be made. This article describes measures to evaluate predictions and the potential improvement in decision making from survival models based on Cox proportional hazards regression. As a motivating case study, the authors consider the prediction of the composite outcome of recurrence or death (the "event") in patients with breast cancer after surgery. They developed a simple Cox regression model with 3 predictors, as in the Nottingham Prognostic Index, in 2982 women (1275 events over 5 years of follow-up) and externally validated this model in 686 women (285 events over 5 years). Improvement in performance was assessed after the addition of progesterone receptor as a prognostic biomarker. The model predictions can be evaluated across the full range of observed follow-up times or for the event occurring by the end of a fixed time horizon of interest. The authors first discuss recommended statistical measures that evaluate model performance in terms of discrimination, calibration, or overall performance. Further, they evaluate the potential clinical utility of the model to support clinical decision making according to a net benefit measure. They provide SAS and R code to illustrate internal and external validation. The authors recommend the proposed set of performance measures for transparent reporting of the validity of predictions from survival models.
Authors 24
-
Affiliation as printed
Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom (D.J.M.)
-
Daniele Giardiello Aachen Netherlands Cancer Institute Department of Biomedical Data Sciences Institute of Biomedicine, Eurac Research
Eurac Research · Leiden University Medical Center · The Netherlands Cancer Institute · Institute for Biomedicine · University of Lübeck
Affiliation as printed
Netherlands Cancer Institute, Amsterdam, the Netherlands, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands, and Institute of Biomedicine, Eurac Research, Affiliated Institute of the University of Lübeck, Bolzano, Italy (D.G.)
-
Ben Van Calster Aachen Department of Biomedical Data Sciences Department of Development and Regeneration
Leiden University Medical Center · KU Leuven
Affiliation as printed
Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands, and Department of Development and Regeneration, Katholieke Universiteit Leuven, Leuven, Belgium (B.V.)
-
Affiliation as printed
School for Public Health and Primary Care, Maastricht University, Maastricht, the Netherlands (L.W.)
-
Leiden University Medical Center
Affiliation as printed
Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands (N.V., E.W.S.)
-
Utrecht University · University Medical Center Utrecht
Affiliation as printed
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands (M.V.)
-
Affiliation as printed
Department of Quantitative Health Sciences, Mayo Clinic, Rochester, Minnesota (T.T.)
-
Leiden University Medical Center
Affiliation as printed
Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands (N.V., E.W.S.)
Cited by 157 stored of 164
No patents citing this paper on Lens.org (checked 2026-10-11).