Deconstructing deep imbalanced regression: a comprehensive review and experimental evaluation

Artificial Intelligence Review, vol. 59

Abstract

Abstract In real-world applications, there is a fundamental problem: the data most critical to predict interesting events, anomalies, and high-stakes outliers are the rarest, while less interesting data is abundant. Although deep learning is deployed specifically for these difficult prediction tasks, data-driven models inevitably fail in underrepresented areas. This discrepancy between the empirical data- and the desired evaluation distribution is equivalent to a target distribution shift. The research field, termed Deep Imbalanced Regression (DIR), has emerged explicitly to address this challenge, which is particularly acute for continuous targets where most conventional classification-based methods are ill-suited. In this paper, we present the first comprehensive review of the DIR landscape, organized around a novel two-axis taxonomy that disentangles challenges along a Data Axis (target distribution shift, continuity, and density) and a Deep-Learning Axis (shared capacity, biased updates, and manifold distortion), where the latter captures a cascading failure mechanism through which deep models systematically neglect underrepresented targets. Within this framework, we systematically categorize and analyze 19 state-of-the-art methods spanning architectural, algorithm-level, and representation learning approaches, and empirically re-evaluate twelve of them with publicly available implementations under controlled, identical conditions. To stress-test generalization across the full target range, we introduce three novel targeted evaluation protocols, Balanced Extrapolation , Bimodal Interpolation , and Blind-Spot Isolation , that expose failure modes hidden by standard benchmarks ( https://github.com/noah-puetz/deconstructing_deep_imbalanced_regression ). Our study underscores the significant impact of imbalance on regression accuracy, offering a conceptual framework and practical benchmarks to catalyze further development of systems capable of capturing the rare as reliably as the common.

Authors 6

1. Noah Christoph Pütz corresponding Aachen

   TH Köln - University of Applied Sciences · Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, Leiden, 2333 CC, Netherlands

   THK-AI Research Cluster, TH Köln, Steinmüllerallee 1, 51643, Gummersbach, Germany

   Toyota Racing, Toyota-Allee 7, 50858, Cologne, Germany

2. Jens Uwe Brandt Aachen

   TH Köln - University of Applied Sciences · Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, Leiden, 2333 CC, Netherlands

   THK-AI Research Cluster, TH Köln, Steinmüllerallee 1, 51643, Gummersbach, Germany

   Toyota Racing, Toyota-Allee 7, 50858, Cologne, Germany

3. Marc Hilbert Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, Leiden, 2333 CC, Netherlands

   Toyota Racing, Toyota-Allee 7, 50858, Cologne, Germany

4. Elena Raponi Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, Leiden, 2333 CC, Netherlands

5. Thomas Bäck Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, Leiden, 2333 CC, Netherlands

6. Thomas Bartz–Beielstein

   TH Köln - University of Applied Sciences

   Affiliation as printed

   THK-AI Research Cluster, TH Köln, Steinmüllerallee 1, 51643, Gummersbach, Germany