A protocol for evaluating robustness to H&E staining variation in computational pathology models

arXiv (Cornell University)

Abstract

Sensitivity to staining variation remains a major barrier to deploying computational pathology (CPath) models as hematoxylin and eosin (H&E) staining varies across laboratories, requiring systematic assessment of how this variability affects model prediction. In this work, we developed a three-step protocol for evaluating robustness to H&E staining variation in CPath models. Step 1: Select reference staining conditions, Step 2: Characterize test set staining properties, Step 3: Apply CPath model(s) under simulated reference staining conditions. Here, we first created a new reference staining library based on the PLISM dataset. As an exemplary use case, we applied the protocol to assess the robustness properties of 306 microsatellite instability (MSI) classification models on the unseen SurGen colorectal cancer dataset (n=738), including 300 attention-based multiple instance learning models trained on the TCGA-COAD/READ datasets across three feature extractors (UNI2-h, H-Optimus-1, Virchow2), alongside six public MSI classification models. Classification performance was measured as AUC, and robustness as the min-max AUC range across four simulated staining conditions (low/high H&E intensity, low/high H&E color similarity). Across models and staining conditions, classification performance ranged from AUC 0.769-0.911 ($Δ$ = 0.142). Robustness ranged from 0.007-0.079 ($Δ$ = 0.072), and showed a weak inverse correlation with classification performance (Pearson r=-0.22, 95% CI [-0.34, -0.11]). Thus, we show that the proposed evaluation protocol enables robustness-informed CPath model selection and provides insight into performance shifts across H&E staining conditions, supporting the identification of operational ranges for reliable model deployment. Code is available at https://github.com/CTPLab/staining-robustness-evaluation .

Authors 8

1. Lydia A. Schönpflug
2. Nikki van den Berg
3. Sonali Andani

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering , University of Basel , Allschwil , Switzerland

   Institute of Medical Genetics and Pathology , University Hospital Basel , Basel , Switzerland

4. Nanda Horeweg Aachen Department of Radiation Oncology

   Leiden University Medical Center

   Affiliation as printed

   Department of Radiation Oncology , Leiden University Medical Center , Leiden , The Netherlands

5. Jurriaan Barkey Wolf Aachen Department of Pathology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pathology , Leiden University Medical Center , Leiden , The Netherlands

6. Tjalling Bosse Aachen Department of Pathology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pathology , Leiden University Medical Center , Leiden , The Netherlands

7. Viktor H. Koelzer

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering , University of Basel , Allschwil , Switzerland

   Institute of Medical Genetics and Pathology , University Hospital Basel , Basel , Switzerland

8. Maxime W. Lafarge

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering , University of Basel , Allschwil , Switzerland

   Institute of Medical Genetics and Pathology , University Hospital Basel , Basel , Switzerland