A step towards understanding why classification helps regression

arXiv (Cornell University)

Abstract

A number of computer vision deep regression approaches report improved results when adding a classification loss to the regression loss. Here, we explore why this is useful in practice and when it is beneficial. To do so, we start from precisely controlled dataset variations and data samplings and find that the effect of adding a classification loss is the most pronounced for regression with imbalanced data. We explain these empirical findings by formalizing the relation between the balanced and imbalanced regression losses. Finally, we show that our findings hold on two real imbalanced image datasets for depth estimation (NYUD2-DIR), and age estimation (IMDB-WIKI-DIR), and on the problem of imbalanced video progress prediction (Breakfast). Our main takeaway is: for a regression task, if the data sampling is imbalanced, then add a classification loss.

Authors 4

1. Silvia L. Pintea Aachen Division of Image Processing (LKEB)

   Leiden University Medical Center · Delft University of Technology

   Affiliation as printed

   Computer Vision Lab , Delft University of Technology

   Division of Image Processing (LKEB) , Leiden University Medical Center

2. Yancong Lin

   Delft University of Technology

   Affiliation as printed

   Intelligent Vehicles Group , Delft University of Technology

3. Jouke Dijkstra Aachen Division of Image Processing (LKEB)

   Leiden University Medical Center

   Affiliation as printed

   Division of Image Processing (LKEB) , Leiden University Medical Center

4. Jan C. van Gemert

   Delft University of Technology

   Affiliation as printed

   Computer Vision Lab , Delft University of Technology