Larger sample sizes are needed when developing a clinical prediction model using machine learning in oncology: methodological systematic review

Journal of Clinical Epidemiology, vol. 180, pp. 111675

Abstract

BACKGROUND AND OBJECTIVES: ). METHODS: , and compared this with the sample size that was used to develop the models. RESULTS: , allowing to precisely estimate the overall risk and minimize overfitting. There was a median deficit of 302 participants with the event (n = 17; range: -21,331 to 2298) when developing the ML models. An additional three out of the 17 studies met the required sample size to precisely estimate the overall risk only. CONCLUSION: . As ML models almost certainly require a larger sample size than regression models, the deficit is likely larger. We recommend that researchers consider and report their sample size and at least meet the minimum sample size required when developing a regression-based model.

Authors 9

1. Biruk Tsegaye

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK. Electronic address: biruk.tsegaye@csm.ox.ac.uk

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK. Electronic address: biruk.tsegaye@csm.ox.ac.uk

2. Kym I E Snell

   National Institute for Health and Care Research · NIHR Birmingham Biomedical Research Centre · University of Birmingham

   Affiliation as printed

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK; Institute of Translational Medicine, National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, United; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

3. Lucinda Archer

   National Institute for Health and Care Research · NIHR Birmingham Biomedical Research Centre · University of Birmingham

   Affiliation as printed

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK; Institute of Translational Medicine, National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, United; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

4. Shona Kirtley

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

5. Richard D Riley

   National Institute for Health and Care Research · NIHR Birmingham Biomedical Research Centre · University of Birmingham

   Affiliation as printed

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK; Institute of Translational Medicine, National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

   Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, B15 2TT, United; National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK

6. Matthew Sperrin

   Manchester Academic Health Science Centre · University of Manchester

   Affiliation as printed

   Division of Imaging, Informatics and Data Science, Manchester Academic Health Science Centre, University of Manchester, Manchester M13 9PL, UK

   Division of Imaging, Informatics and Data Science, University of Manchester, Manchester Academic Health Science Centre, Manchester, M13 9PL, United Kingdom

7. Ben Van Calster Aachen

   Leiden University · Leiden University Medical Center · KU Leuven

   Affiliation as printed

   Department of Development and Regeneration, KU Leuven, Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands; Leuven Unit for Health Technology Assessment Research (LUHTAR), KU Leuven, Leuven, Belgium

   Department of Development and Regeneration, KU Leuven, Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands; Leuven Unit for Health Technology Assessment Research (LUHTAR), KU Leuven, Leuven, Belgium

8. Professor Gary S. Collins

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

9. Paula Dhiman

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK