Automated machine learning: past, present and future

Artificial Intelligence Review, vol. 57

Abstract

Abstract Automated machine learning (AutoML) is a young research area aiming at making high-performance machine learning techniques accessible to a broad set of users. This is achieved by identifying all design choices in creating a machine-learning model and addressing them automatically to generate performance-optimised models. In this article, we provide an extensive overview of the past and present, as well as future perspectives of AutoML. First, we introduce the concept of AutoML, formally define the problems it aims to solve and describe the three components underlying AutoML approaches: the search space, search strategy and performance evaluation. Next, we discuss hyperparameter optimisation (HPO) techniques commonly used in AutoML systems design, followed by providing an overview of the neural architecture search, a particular case of AutoML for automatically generating deep learning models. We further review and compare available AutoML systems. Finally, we provide a list of open challenges and future research directions. Overall, we offer a comprehensive overview for researchers and practitioners in the area of machine learning and provide a basis for further developments in AutoML.

Authors 7

1. Mitra Baratchi corresponding Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

2. Can Wang corresponding Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

3. Steffen Limmer corresponding
   Affiliation as printed

   Honda Research Institute Europe, Offenbach, Germany

4. Jan N. van Rijn corresponding Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

5. Holger H. Hoos corresponding Aachen Chair for AI Methodology (AIM) Leiden Institute of Advanced Computer Science

   Leiden University · RWTH Aachen University

   Affiliation as printed

   Chair for AI Methodology (AIM), RWTH Aachen University, Aachen, Germany

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

6. Thomas Bäck corresponding Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

7. Markus Olhofer corresponding
   Affiliation as printed

   Honda Research Institute Europe, Offenbach, Germany