Counterfactual Diffusion Models for Interpretable Explanations of Artificial Intelligence Models in Pathology
bioRxiv (Cold Spring Harbor Laboratory)
Abstract
Abstract Deep learning can extract predictive and prognostic biomarkers from histopathology whole slide images. However, explainable artificial intelligence approaches widely used in digital pathology, such as attention heatmaps and class activation mapping, offer only limited interpretability regarding the features captured by classifiers. Here, we present MoPaDi (Morphing histoPathology Diffusion), a framework for generating counterfactual explanations for histopathology images that reveal which morphological or style features drive classifier predictions. MoPaDi combines diffusion autoencoders with task-specific multiple instance learning classifiers to manipulate images and flip predictions by modifying relevant features. We evaluated the framework on multiple datasets spanning colorectal, breast, liver, and lung cancers, including tissue type, cancer subtype, and biomarker (microsatellite instability) classification tasks. We assessed counterfactual explanations through quantitative analyses, pathologists’ evaluations, and independent foundation model-based classifiers. We found that MoPaDi was able to generate realistic counterfactual histopathology images, enabling pathologists to identify morphological features associated with the change in model predictions. Unlike conventional reviews of highly attended regions typical in digital pathology, MoPaDi explanations enabled pathologists to directly identify morphological features driving the classifier’s predictions from a limited number of top-contributing tiles. Consistent with the literature, our biomarker classifier associated high microsatellite instability with mucinous differentiation, glandular patterns, and lymphocytic infiltration. Furthermore, MoPaDi revealed that changes in classifier predictions were mainly driven by morphological alterations rather than staining differences. Overall, MoPaDi is a practical framework for counterfactual explanations in computational pathology that reveals model-specific drivers of classification and increases trust in deep learning models.
Authors 11
-
University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Germany
-
University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
University of Augsburg · University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Bavarian Cancer Research Center (BZKF), Augsburg, Germany
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Pathology, Faculty of Medicine, University of Augsburg, Augsburg, Germany
Bavarian Cancer Research Center (BZKF)
EKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TU Dresden
Faculty of Medicine, University of Augsburg
-
Johannes Gutenberg University Mainz · University Medical Center of the Johannes Gutenberg University Mainz
Affiliation as printed
Institute of Pathology, University Medical Center of the Johannes Gutenberg University Mainz, Germany
-
University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
University of Chicago · Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Section of Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, IL, USA
Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Germany
-
Universitätsklinikum Aachen · University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden · RWTH Aachen University
Affiliation as printed
Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Germany
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
-
Jakob Nikolas Kather corresponding
University of Leeds · University Hospital Heidelberg · National Center for Tumor Diseases · University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Department of Medicine I, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
Pathology & Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds, United Kingdom
Cited by 7 stored of 7
7 results
No patents citing this paper on Lens.org (checked 2026-10-06).