How to develop, validate, and update clinical prediction models using multinomial logistic regression

Journal of Clinical Epidemiology, vol. 174, pp. 111481

Abstract

OBJECTIVES: Multicategory prediction models (MPMs) can be used in health care when the primary outcome of interest has more than two categories. The application of MPMs is scarce, possibly due to added methodological complexities compared to binary outcome models. We provide a guide of how to develop, validate, and update clinical prediction models based on multinomial logistic regression. STUDY DESIGN AND SETTING: We present guidance and recommendations based on recent methodological literature, illustrated by a previously developed and validated MPM for treatment outcomes in rheumatoid arthritis. Prediction models using multinomial logistic regression can be developed for nominal outcomes, but also for ordinal outcomes. This article is intended to supplement existing general guidance on prediction model research. RESULTS: This guide is split into three parts: 1) outcome definition and variable selection, 2) model development, and 3) model evaluation (including performance assessment, internal and external validation, and model recalibration). We outline how to evaluate and interpret the predictive performance of MPMs. R code is provided. CONCLUSION: We recommend the application of MPMs in clinical settings where the prediction of a multicategory outcome is of interest. Future methodological research could focus on MPM-specific considerations for variable selection and sample size criteria for external validation.

Authors 6

1. Celina K Gehringer corresponding

   Manchester Academic Health Science Centre · University of Manchester · Centre for Epidemiology Versus Arthritis

   Affiliation as printed

   Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Division of Musculoskeletal and Dermatological Sciences, University of Manchester, Manchester, UK; Centre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK. Electronic address: celina.gehringer@postgrad.manchester.ac.uk

2. Glen Philip Martin

   University of Manchester

   Affiliation as printed

   Division of Informatics, Imaging and Data Sciences, Centre for Health Informatics, University of Manchester, Manchester, UK

3. Ben Van Calster Aachen

   Leiden University · Leiden University Medical Center · KU Leuven

   Affiliation as printed

   Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands; Department of Development & Regeneration, KU Leuven, Leuven, Belgium

4. Kimme Leanne Hyrich

   Manchester Academic Health Science Centre · University of Manchester · Manchester University NHS Foundation Trust · NIHR Manchester Biomedical Research Centre · Centre for Epidemiology Versus Arthritis

   Affiliation as printed

   Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Division of Musculoskeletal and Dermatological Sciences, University of Manchester, Manchester, UK; NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK

5. Suzanne M. M. Verstappen

   Manchester Academic Health Science Centre · University of Manchester · Manchester University NHS Foundation Trust · NIHR Manchester Biomedical Research Centre · Centre for Epidemiology Versus Arthritis

   Affiliation as printed

   Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Division of Musculoskeletal and Dermatological Sciences, University of Manchester, Manchester, UK; NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK

6. Jamie C. Sergeant

   Manchester Academic Health Science Centre · University of Manchester · Centre for Epidemiology Versus Arthritis

   Affiliation as printed

   Centre for Epidemiology Versus Arthritis, Centre for Musculoskeletal Research, Division of Musculoskeletal and Dermatological Sciences, University of Manchester, Manchester, UK; Centre for Biostatistics, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK