Optimizing With Low Budgets: A Comparison on the Black-Box Optimization Benchmarking Suite and OpenAI Gym

IEEE Transactions on Evolutionary Computation, vol. 29, pp. 91–101

Abstract

The growing ubiquity of machine learning (ML) has led it to enter various areas of computer science, including black-box optimization (BBO). Recent research is particularly concerned with Bayesian optimization (BO). BO-based algorithms are popular in the ML community, as they are used for hyperparameter optimization and more generally for algorithm configuration. However, their efficiency decreases as the dimensionality of the problem and the budget of evaluations increase. Meanwhile, derivative-free optimization methods have evolved independently in the optimization community. Therefore, we urge to understand whether cross-fertilization is possible between the two communities, ML and BBO, i.e., whether algorithms that are heavily used in ML also work well in BBO and vice versa. Comparative experiments often involve rather small benchmarks and show visible problems in the experimental setup, such as poor initialization of baselines, overfitting due to problem-specific setting of hyperparameters, and low statistical significance. With this paper, we update and extend a comparative study presented by Hutter et al. in 2013. We compare BBO tools for ML with more classical heuristics, first on the well-known BBOB benchmark suite from the COCO environment and then on Direct Policy Search for OpenAI Gym, a reinforcement learning benchmark. Our results confirm that BO-based optimizers perform well on both benchmarks when budgets are limited, albeit with a higher computational cost, while they are often outperformed by algorithms from other families when the evaluation budget becomes larger. We also show that some algorithms from the BBO community perform surprisingly well on ML tasks.

Authors 5

1. Elena Raponi Aachen

   Leiden University · Technical University of Munich

   Affiliation as printed

   Meta AI Research, Paris, France

   LIACS, Leiden University, Leiden, The Netherlands

   Technical University of Munich, Munich, Germany

2. Nathanaël Carraz Rakotonirina

   University of Antananarivo · Universitat Pompeu Fabra

   Affiliation as printed

   LIMA, Université d’Antananarivo, Antananarivo, Madagascar

   Universitat Pompeu Fabra, Barcelona, Spain

3. Jérémy Rapin
   Affiliation as printed

   Meta AI Research, Paris, France

4. Carola Doerr

   Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   LIP6, CNRS, Sorbonne Université, Paris, France

5. Olivier Teytaud
   Affiliation as printed

   Meta AI Research, Paris, France