TRIPOD statement: a preliminary pre-post analysis of reporting and methods of prediction models

BMJ Open, vol. 10, pp. e041537

Abstract

OBJECTIVES: To assess the difference in completeness of reporting and methodological conduct of published prediction models before and after publication of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. METHODS: In the seven general medicine journals with the highest impact factor, we compared the completeness of the reporting and the quality of the methodology of prediction model studies published between 2012 and 2014 (pre-TRIPOD) with studies published between 2016 and 2017 (post-TRIPOD). For articles published in the post-TRIPOD period, we examined whether there was improved reporting for articles (1) citing the TRIPOD statement, and (2) published in journals that published the TRIPOD statement. RESULTS: A total of 70 articles was included (pre-TRIPOD: 32, post-TRIPOD: 38). No improvement was seen for the overall percentage of reported items after the publication of the TRIPOD statement (pre-TRIPOD 74%, post-TRIPOD 76%, 95% CI of absolute difference: -4% to 7%). For the individual TRIPOD items, an improvement was seen for 16 (44%) items, while 3 (8%) items showed no improvement and 17 (47%) items showed a deterioration. Post-TRIPOD, there was no improved reporting for articles citing the TRIPOD statement, nor for articles published in journals that published the TRIPOD statement. The methodological quality improved in the post-TRIPOD period. More models were externally validated in the same article (absolute difference 8%, post-TRIPOD: 39%), used measures of calibration (21%, post-TRIPOD: 87%) and discrimination (9%, post-TRIPOD: 100%), and used multiple imputation for handling missing data (12%, post-TRIPOD: 50%). CONCLUSIONS: Since the publication of the TRIPOD statement, some reporting and methodological aspects have improved. Prediction models are still often poorly developed and validated and many aspects remain poorly reported, hindering optimal clinical application of these models. Long-term effects of the TRIPOD statement publication should be evaluated in future studies.

Authors 10

1. Amir H. Zamanipoor Najafabadi corresponding Aachen Department of Neurosurgery, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Neurosurgery, Leiden University Medical Center, Leiden, The Netherlands

2. Chava L. Ramspek Aachen Department of Clinical Epidemiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

3. Friedo Wilhelm Dekker Aachen Department of Clinical Epidemiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

4. Pauline Heus

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center (UMC) Utrecht, Utrecht University, Utrecht, The Netherlands

5. Lotty Hooft

   Utrecht University · University Medical Center Utrecht

   Affiliation as printed

   Dutch Cochrane Centre (DCC), Julius Center for Health Sciences and Primary Care, University Medical Centre (UMC) Utrecht, Utrecht University, Utrecht, The Netherlands

6. Karel G.M. Moons
   Affiliation as printed

   Department of Epidemiology, Julius Center for Health Sciences and Primary Care, Utrecht, The Netherlands

7. Wilco C. Peul Aachen Department of Neurosurgery, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Neurosurgery, Leiden University Medical Center, Leiden, The Netherlands

   Department of Neurosurgery, The Hague Medical Center, The Hague, The Netherlands

8. Professor Gary S. Collins

   University of Oxford

   Affiliation as printed

   NDORMS, Oxford University, Oxford, UK

9. Ewout Willem Steyerberg Aachen Department of Medical Statistics

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Statistics, Leiden University Medical Center, Leiden, The Netherlands

10. Merel van Diepen Aachen Department of Clinical Epidemiology

    Leiden University Medical Center

    Affiliation as printed

    Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands