A Systematic Review of Machine Learning-based Prognostic Models for Acute Pancreatitis: Towards Improving Methods and Reporting Quality
Abstract
Abstract Background An accurate prognostic tool is essential to aid clinical decision making (e.g., patient triage) and to advance personalized medicine. However, such prognostic tool is lacking for acute pancreatitis (AP). Increasingly machine learning (ML) techniques are being used to develop high-performing prognostic models in AP. However, methodologic and reporting quality has received little attention . High-quality reporting and study methodology are critical to model validity, reproducibility, and clinical implementation. In collaboration with content experts in ML methodology, we performed a systematic review critically appraising the quality of methodology and reporting of recently published ML AP prognostic models. Methods Using a validated search strategy, we identified ML AP studies from the databases MEDLINE, PubMed, and EMBASE published between January 2021 and December 2023. Eligibility criteria included all retrospective or prospective studies that developed or validated new or existing ML models in patients with AP that predicted an outcome following an episode of AP. Meta-analysis was considered if there was homogeneity in the study design and in the type of outcome predicted. For risk of bias (ROB) assessment, we used the Prediction Model Risk of Bias Assessment Tool (PROBAST). Quality of reporting was assessed using the Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis – Artificial Intelligence (TRIPOD+AI) statement that defines standards for 27 items that should be reported in publications using ML prognostic models. Results The search strategy identified 6480 publications of which 30 met the eligibility criteria. Studies originated from China (22), U.S (4), and other (4). All 30 studies developed a new ML model and none sought to validate an existing ML model, producing a total of 39 new ML models. AP severity (23/39) or mortality (6/39) were the most common outcomes predicted. The mean area-under-the-curve for all models and endpoints was 0.91 (SD 0.08). The ROB was high for at least one domain in all 39 models, particularly for the analysis domain (37/39 models). Steps were not taken to minimize over-optimistic model performance in 27/39 models. Due to heterogeneity in the study design and in how the outcomes were defined and determined, meta-analysis was not performed. Studies reported on only 15/27 items from TRIPOD+AI standards, with only 7/30 justifying sample size and 13/30 assessing data quality. Other reporting deficiencies included omissions regarding human-AI interaction (28/30), handling low-quality or incomplete data in practice (27/30), sharing analytical codes (25/30), study protocols (25/30) and reporting source data (19/30),. Discussion There are significant deficiencies in the methodology and reporting of recently published ML based prognostic models in AP patients. These undermine the validity, reproducibility and implementation of these prognostic models despite their promise of superior predictive accuracy. Funding none Registration Research Registry ( reviewregistry1727 )
Authors 34
-
Affiliation as printed
Weill Cornell Medical College, Division of Gastroenterology and Hepatology
Weill Cornell Medical College: Weill Cornell Medicine
-
Affiliation as printed
Weill Cornell Medical College, Division of Gastroenterology and Hepatology
Weill Cornell Medicine
-
The Ohio State University Wexner Medical Center · Ohio University · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
Ohio University College of Osteopathic Medicine: Ohio University Heritage College of Osteopathic Medicine
-
Northeast Ohio Medical University
Affiliation as printed
Northeast Ohio Medical School
Northeast Ohio Medical University College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University OSUMC: The Ohio State University Wexner Medical Center
-
Affiliation as printed
Weill Cornell Medical College, Division of Gastroenterology and Hepatology
Weill Cornell Medical College: Weill Cornell Medicine
-
Affiliation as printed
Weill Cornell Medical College, Division of Gastroenterology and Hepatology
Weill Cornell Medical College: Weill Cornell Medicine
-
Affiliation as printed
Rheinisch-Westfälische Technische Hochschule Aachen University, Division of Process and Data Science
Rheinisch-Westfälische Technische Hochschule Aachen: Rheinisch-Westfalische Technische Hochschule Aachen
-
Affiliation as printed
Rheinisch-Westfälische Technische Hochschule Aachen University, Division of Process and Data Science
Rheinisch-Westfälische Technische Hochschule Aachen: Rheinisch-Westfalische Technische Hochschule Aachen
-
Massachusetts Institute of Technology
Affiliation as printed
Massachusetts Institute of Technology, Laboratory for Computational Physiology
Massachusetts Institute of Technology
-
Massachusetts Institute of Technology
Affiliation as printed
Massachusetts Institute of Technology, Laboratory for Computational Physiology
Massachusetts Institute of Technology
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Bioinformatics
The Ohio State University OSUMC: The Ohio State University Wexner Medical Center
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Bioinformatics
The Ohio State University Medical Center: The Ohio State University Wexner Medical Center
-
Affiliation as printed
Center for Systems Immunology, Benaroya Research Institute at Virginia Mason, Seattle, Washington
Benaroya Research Institute at Virginia Mason
-
Affiliation as printed
Center for Interventional Immunology, Benaroya Research Institute at Virginia Mason, Seattle, Washington
Benaroya Research Institute at Virginia Mason
-
University of Southern California
Affiliation as printed
University of Southern California, Division of Gastroenterology
University of Southern California Keck School of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Emergency Medicine
The Ohio State University College of Medicine
-
University of Illinois Chicago
Affiliation as printed
University of Illinois at Chicago, Division of Gastroenterology
University of Illinois Chicago College of Medicine
-
University of Pittsburgh · University of Pittsburgh Medical Center
Affiliation as printed
University of Pittsburgh Medical Center, Division of Gastroenterology
: University of Pittsburgh Department of Medicine
-
University of Southern California
Affiliation as printed
University of Southern California, Division of Emergency Medicine
University of Southern California Keck School of Medicine
-
University of Pittsburgh · University of Pittsburgh Medical Center
Affiliation as printed
University of Pittsburgh Medical Center, Division of Emergency Medicine
University of Pittsburgh School of Medicine
-
Affiliation as printed
University of Kentucky, Department of Medicine
University of Kentucky College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University Wexner Medical Center
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
Johns Hopkins University · Johns Hopkins Medicine · Johns Hopkins Hospital
Affiliation as printed
Johns Hopkins Medical Center, Division of Gastroenterology
Johns Hopkins Medicine
-
Affiliation as printed
University of Auckland, Surgical and Translational Research Centre
University of Auckland
-
University of Pennsylvania Health System · University of Pennsylvania
Affiliation as printed
University of Pennsylvania, Division of Hospital Medicine and Penn Medicine Center for Evidence-based Practice
University of Pennsylvania Perelman School of Medicine
-
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
The Ohio State University College of Medicine
-
Massachusetts Institute of Technology
Affiliation as printed
Beth Israel Medical Center, Division of Critical Care
Massachusetts Institute of Technology, Laboratory for Computational Physiology
Massachusetts Institute of Technology
-
Peter Junwoo Lee corresponding
The Ohio State University Wexner Medical Center · The Ohio State University
Affiliation as printed
Ohio State University Wexner Medical Center, Division of Gastroenterology and Hepatology
OSUMC: The Ohio State University Wexner Medical Center
Cited by 1 stored of 1
1 result
No patents citing this paper on Lens.org (checked 2026-10-06).