Reporting of prognostic clinical prediction models based on machine learning methods in oncology needs to be improved

Journal of Clinical Epidemiology, vol. 138, pp. 60–72

Abstract

OBJECTIVE: Evaluate the completeness of reporting of prognostic prediction models developed using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE and Embase databases between 01/01/2019 and 05/09/2019, for non-imaging studies developing a prognostic clinical prediction model using machine learning methods (as defined by primary study authors) in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement to assess the reporting quality of included publications. We described overall reporting adherence of included publications and by each section of TRIPOD. RESULTS: Sixty-two publications met the inclusion criteria. 48 were development studies and 14 were development with validation studies. 152 models were developed across all publications. Median adherence to TRIPOD reporting items was 41% [range: 10%-67%] and at least 50% adherence was found in 19% (n=12/62) of publications. Adherence was lower in development only studies (median: 38% [range: 10%-67%]); and higher in development with validation studies (median: 49% [range: 33%-59%]). CONCLUSION: Reporting of clinical prediction models using machine learning in oncology is poor and needs urgent improvement, so readers and stakeholders can appraise the study methods, understand study findings, and reduce research waste.

Authors 12

1. Paula Dhiman corresponding

   University of Oxford · NIHR Oxford Biomedical Research Centre · Oxford University Hospitals NHS Trust

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom. Electronic address: paula.dhiman@csm.ox.ac.uk

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

   NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom

2. Jie Ma

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

3. Constanza L. Andaur Navarro

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

4. Benjamin Speich

   University of Basel · University Hospital of Basel · University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK; Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

   Department of Clinical Research, Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland

5. Garrett Scott Bullock

   University of Oxford

   Affiliation as printed

   Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK

6. Johanna AAG Damen

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

7. Shona Kirtley

   University of Oxford

   Affiliation as printed

   Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

8. Lotty Hooft

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

9. Richard D Riley

   Keele University

   Affiliation as printed

   Centre for Prognosis Research, School of Medicine, Keele University, Staffordshire, UK. ST5 5BG

10. Ben Van Calster Aachen Department of Biomedical Data Sciences

    Leiden University Medical Center · KU Leuven

    Affiliation as printed

    Department of Development and Regeneration, KU Leuven, Leuven, Belgium.; Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands.; EPI-centre, KU Leuven, Leuven, Belgium

    Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands

    Department of Development and Regeneration, KU Leuven, Leuven, Belgium

    EPI-centre, KU Leuven, Leuven, Belgium

11. Karel G.M. Moons

    Utrecht University · University Medical Center Utrecht

    Affiliation as printed

    Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

12. Professor Gary S. Collins

    University of Oxford · NIHR Oxford Biomedical Research Centre

    Affiliation as printed

    Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom

    Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, OX3 7LD, UK

    NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom