A protocol for evaluating robustness to H&E staining variation in computational pathology models

Journal of Pathology Informatics, vol. 22, pp. 100702

Abstract

Sensitivity to staining variation remains a major barrier to deploying computational pathology (CPath) models as hematoxylin and eosin (H&E) staining varies across labs, requiring systematic assessment of how this variability affects model prediction. Yet, current evaluation practices rarely attribute performance changes to specific, measurable staining properties, limiting interpretable robustness assessment under controlled conditions. We introduce a three-step protocol that explicitly simulates H&E staining variation toward known, measurable targets, supporting controlled assessment of model performance shifts. Step 1: Select reference staining conditions, Step 2: Characterize test set staining properties, Step 3: Apply CPath model(s) under simulated reference staining conditions. Here, we construct a new reference staining library based on the PLISM dataset. As a use case, we evaluate 306 microsatellite instability (MSI) classification models on the SurGen colorectal cancer dataset ( n = 738), including 300 attention-based multiple instance learning models trained on TCGA-COAD/READ across three feature extractors (UNI2-h, H-Optimus-1, and Virchow2), and six public models. Classification performance was measured as AUC, and robustness as the min-max AUC range across four simulated staining conditions (low/high H&E intensity, low/high H&E color similarity). Across models and staining conditions, classification performance ranged from AUC 0.769–0.911 (Δ = 0.142). Robustness ranged from 0.007 to 0.079 (Δ = 0.072), and showed a weak inverse correlation with classification performance (Pearson r = −0.28, 95% CI [−0.39, −0.17]). Thus, using MSI classification in colorectal cancer as a use case, we demonstrate that the proposed evaluation protocol enables robustness-informed CPath model selection and provides insight into how H&E staining conditions affect model performance, with the potential to support reliable deployment across clinical settings. Code and reference library are publicly available at https://huggingface.co/datasets/CTPLab-DBE-UniBas/staining-robustness-evaluation and https://github.com/CTPLab/staining-robustness-evaluation .

Authors 8

1. Lydia A. Schoenpflug

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland

   Institute of Medical Genetics and Pathology, University Hospital Basel, Basel, Switzerland

2. Nikki van den Berg Aachen Department of Pathology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pathology, Leiden University Medical Center, Leiden, the Netherlands

3. Sonali Andani

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland

   Institute of Medical Genetics and Pathology, University Hospital Basel, Basel, Switzerland

4. Nanda Horeweg Aachen Department of Radiation Oncology

   Leiden University Medical Center

   Affiliation as printed

   Department of Radiation Oncology, Leiden University Medical Center, Leiden, the Netherlands

5. Jurriaan Barkey Wolf Aachen Department of Pathology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pathology, Leiden University Medical Center, Leiden, the Netherlands

6. Tjalling Bosse Aachen Department of Pathology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pathology, Leiden University Medical Center, Leiden, the Netherlands

7. Viktor H. Koelzer

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland

   Institute of Medical Genetics and Pathology, University Hospital Basel, Basel, Switzerland

8. Maxime W. Lafarge corresponding

   University of Basel · University Hospital of Basel

   Affiliation as printed

   Department of Biomedical Engineering, University of Basel, Allschwil, Switzerland

   Institute of Medical Genetics and Pathology, University Hospital Basel, Basel, Switzerland