Efficient AutoML via Combinational Sampling

IEEE Symposium Series on Computational Intelligence (SSCI), pp. 01–10

Abstract

Automated machine learning (AutoML) aims to automatically produce the best machine learning pipeline, i.e., a sequence of operators and their optimized hyperparameter settings, to maximize the performance of an arbitrary machine learning problem. Typically, AutoML based Bayesian optimization (BO) approaches convert the AutoML optimization problem into a Hyperparameter Optimization (HPO) problem, where the choice of algorithms is modeled as an additional categorical hyperparameter. In this way, algorithms and their local hyperparameters are referred to as the same level. Consequently, this approach makes the resulting initial sampling less robust. In this study, we describe a first attempt to formulate the AutoML optimization problem as its nature instead of transfer it into a HPO problem. To take advantage of this paradigm, we propose a novel initial sampling approach to maximize the coverage of the AutoML search space to help BO construct a robust surrogate model. We experiment with 2 independent scenarios of AutoML with 2 operators and 6 operators over 117 benchmark datasets. Results of our experiments demonstrate that the performance of BO significantly improved by using our sampling approach.

Authors 5

1. Duc Anh Nguyen Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden University,Leiden Institute of Advanced Computer Science (LIACS),The Netherlands

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands

2. Anna V. Kononova Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden University,Leiden Institute of Advanced Computer Science (LIACS),The Netherlands

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands

3. Stefan Menzel

   Honda (Germany)

   Affiliation as printed

   Honda Research Institute Europe (HRI-EU), Offenbach/Main,Germany

   Honda Research Institute Europe (HRI-EU), Offenbach/Main, Germany

4. Bernhard Sendhoff

   Honda (Germany)

   Affiliation as printed

   Honda Research Institute Europe (HRI-EU), Offenbach/Main,Germany

   Honda Research Institute Europe (HRI-EU), Offenbach/Main, Germany

5. Thomas Bäck Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden University,Leiden Institute of Advanced Computer Science (LIACS),The Netherlands

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, The Netherlands