The Impact of Digital Histopathology Batch Effect on Deep Learning Model Accuracy and Bias

bioRxiv (Cold Spring Harbor Laboratory)

Abstract

Abstract The Cancer Genome Atlas (TCGA) is one of the largest biorepositories of digital histology. Deep learning (DL) models have been trained on TCGA to predict numerous features directly from histology, including survival, gene expression patterns, and driver mutations. However, we demonstrate that these features vary substantially across tissue submitting sites in TCGA for over 3,000 patients with six cancer subtypes. Additionally, we show that histologic image differences between submitting sites can easily be identified with DL. This site detection remains possible despite commonly used color normalization and augmentation methods, and we quantify the digital image characteristics constituting this histologic batch effect. As an example, we show that patient ethnicity within the TCGA breast cancer cohort can be inferred from histology due to site-level batch effect, which must be accounted for to ensure equitable application of DL. Batch effect also leads to overoptimistic estimates of model performance, and we propose a quadratic programming method to guide validation that abrogates this bias.

Authors 13

1. Frederick Matthew Howard

   University of Chicago

   Affiliation as printed

   Department of Medicine, University of Chicago

2. James M. Dolezal

   University of Chicago

   Affiliation as printed

   Department of Medicine, University of Chicago

3. Sara E. Kochanny

   University of Chicago

   Affiliation as printed

   Department of Medicine, University of Chicago

4. Jefree J. Schulte

   University of Chicago

   Affiliation as printed

   Department of Pathology, University of Chicago

5. Heather Chen

   University of Chicago

   Affiliation as printed

   Department of Pathology, University of Chicago

6. Lara Rosaline Heij Aachen Department of Surgery and Transplantation Institute of Pathology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Surgery and Transplantation, University Hospital RWTH Aachen

   Institute of Pathology, University Hospital RWTH Aachen

7. Dezheng Huo

   University of Chicago

   Affiliation as printed

   Department of Public Health Sciences, University of Chicago

8. Rita Nanda

   University of Chicago

   Affiliation as printed

   Department of Medicine, University of Chicago

9. Olufunmilayo Ibironke Olopade

   University of Chicago

   Affiliation as printed

   Department of Medicine, University of Chicago

10. Jakob Nikolas Kather Aachen Department of Medicine III

    University of Leeds · University Hospital Heidelberg · National Center for Tumor Diseases · Universitätsklinikum Aachen · RWTH Aachen University

    Affiliation as printed

    Department of Medicine III, University Hospital RWTH Aachen

    Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg

    Pathology and Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds

    Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds

11. Nicole A. Cipriani

    University of Chicago

    Affiliation as printed

    Department of Pathology, University of Chicago

12. Robert L. Grossman

    University of Chicago

    Affiliation as printed

    Department of Medicine, University of Chicago

13. Alexander T. Pearson

    University of Chicago

    Affiliation as printed

    Department of Medicine, University of Chicago