Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy

Scientific Reports, vol. 12, pp. 4829

Abstract

Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N = 29,506 gastroscopy and N = 18,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.

Authors 11

1. Lukas Buendgens Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

2. Didem Çifçi Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

3. Narmin Ghaffari Laleh Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

4. Marko van Treeck Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

5. Maria Teresa Koenen Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

   Department of Medicine, Rhein-Maas-Klinikum, Würselen, Germany

6. Henning Wolfgang Zimmermann Aachen Department of Medicine III

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

7. Till Herbold Aachen Department of Visceral Surgery and Transplantation

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Visceral Surgery and Transplantation, University Hospital RWTH Aachen, Aachen, Germany

8. Thomas Joachim Lux

   University of Würzburg · Universitätsklinikum Würzburg

   Affiliation as printed

   Interventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital Würzburg, Würzburg, Germany

9. Alexander Hann

   University of Würzburg · Universitätsklinikum Würzburg

   Affiliation as printed

   Interventional and Experimental Endoscopy (InExEn), Internal Medicine II, University Hospital Würzburg, Würzburg, Germany

10. Christian Trautwein Aachen Department of Medicine III

    Universitätsklinikum Aachen · RWTH Aachen University

    Affiliation as printed

    Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany

11. Jakob Nikolas Kather corresponding Aachen Department of Medicine III

    University of Leeds · University Hospital Heidelberg · National Center for Tumor Diseases · Universitätsklinikum Aachen · RWTH Aachen University

    Affiliation as printed

    Department of Medicine III, University Hospital RWTH Aachen, Pauwelsstrasse 30, 52074, Aachen, Germany. jkather@ukaachen.de

    Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK. jkather@ukaachen.de

    Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany. jkather@ukaachen.de

    Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK

    Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany