Feature analysis for proper intensity scaling and feature distinction in class activation maps

Knowledge-Based Systems, vol. 328, pp. 114209

Abstract

Understanding the decisions of deep learning (DL) models is crucial for their acceptance in risk-sensitive applications. Class activation maps (CAMs) are commonly used in image analysis to visualize model reasoning by generating attention maps where signal intensities represent contributions to outputs. However, existing CAM algorithms focus on optimal weight design and salient feature layer selection, neglecting two key limitations in population-level interpretation: (1) lack of an appropriate intensity scale for proper interpretation and quantitative analysis, and (2) inability to identify which features within selected layers predominantly drive model reasoning. These gaps can lead to miscorrelations between CAMs and model outputs, causing erroneous interpretations, while restricting CAMs to case-specific visual inspections. We propose a framework to statistically analyze DL-extracted features at a population level, determinizing feature contributions for global intensity scaling and within-layer feature distinction. The global intensity scale standardizes CAMs, achieving high correlation coefficients (R) with model outputs. Within-layer feature distinction identifies overfitting, confounding factors, outliers, redundancies, and principal features. Applied to eight datasets, including five medical imaging datasets, this framework improved Rs between CAMs and outputs by 10.7–64.2%, achieving near 100% consistency. Furthermore, distinguishing principal features (5–25% in the selected layer) produced CAMs of equal quality while maintaining model output accuracy. This method, relying on minimal assumptions, enhances CAM-model consistency and broadens CAM applications by enabling standardized interpretation and deeper feature-level insights.

Authors 6

1. Yanli Li Aachen Division of Image Processing Department of Radiology, Leiden University Medical Center

   Leiden University · Leiden University Medical Center

   Affiliation as printed

   Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands

2. Denis P. Shamonin Aachen Division of Image Processing Department of Radiology, Leiden University Medical Center

   Leiden University · Leiden University Medical Center

   Affiliation as printed

   Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands

3. Tahereh Hassanzadeh Aachen Division of Image Processing Department of Radiology, Leiden University Medical Center

   Leiden University · Leiden University Medical Center

   Affiliation as printed

   Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands

4. M. Reijnierse Aachen Department of Radiology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands

5. Annette H M van der Helm–van Mil
6. Berend C. Stoel corresponding Aachen Division of Image Processing Department of Radiology, Leiden University Medical Center

   Leiden University · Leiden University Medical Center

   Affiliation as printed

   Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands