Minimum Sample Size for Developing a Multivariable Prediction Model using Multinomial Logistic Regression

Research Explorer (The University of Manchester)

Abstract

Multinomial logistic regression models allow one to predict the risk of a categorical outcome with more than 2 categories. When developing such a model, researchers should ensure the number of participants (n) is appropriate relative to the number of events (E.k) and the number of predictor parameters (p.k) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes. The first criteria aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R2 of distinct one-to-one logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R2 of the multinomial logistic regression. We tested the performance of the proposed criteria (i) through a simulation study, and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) are natural extensions from previously proposed criteria for binary outcomes. We illustrate how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.

Authors 7

1. Alexander Pate

   Manchester Academic Health Science Centre · University of Manchester

   Affiliation as printed

   . Division of Informatics, Imaging and Data Science , Faculty of Biology , Medicine and Health , University of Manchester , Manchester Academic Health Science Centre , Manchester , UK

2. Richard D Riley

   Keele University

   Affiliation as printed

   . Centre for Prognosis Research , School of Medicine , Keele University , Staffordshire , UK

3. Professor Gary S. Collins

   Nuffield Orthopaedic Centre · John Radcliffe Hospital · NIHR Oxford Biomedical Research Centre

   Affiliation as printed

   . Centre for Statistics in Medicine , Nuffield Department of Orthopaedics , Rheumatology and Musculoskeletal Sciences , University of Oxford , Oxford , UK

   . NIHR Oxford Biomedical Research Centre , John Radcliffe Hospital , Oxford , UK

4. Maarten van Smeden Aachen Department of Clinical Epidemiology

   Leiden University Medical Center · University Medical Center Utrecht

   Affiliation as printed

   . Department of Clinical Epidemiology , Leiden University Medical Center Leiden , Netherlands

   . Julius Center for Health Sciences , University Medical Center Utrecht , Utrecht University , Utrecht , Netherlands

5. Ben Van Calster
6. Joie Ensor

   Keele University

   Affiliation as printed

   . Centre for Prognosis Research , School of Medicine , Keele University , Staffordshire , UK

7. Glen Philip Martin

   Manchester Academic Health Science Centre · University of Manchester

   Affiliation as printed

   . Division of Informatics, Imaging and Data Science , Faculty of Biology , Medicine and Health , University of Manchester , Manchester Academic Health Science Centre , Manchester , UK