Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB

arXiv (Cornell University)

Abstract

Bayesian Optimization (BO) is a class of black-box, surrogate-based heuristics that can efficiently optimize problems that are expensive to evaluate, and hence admit only small evaluation budgets. BO is particularly popular for solving numerical optimization problems in industry, where the evaluation of objective functions often relies on time-consuming simulations or physical experiments. However, many industrial problems depend on a large number of parameters. This poses a challenge for BO algorithms, whose performance is often reported to suffer when the dimension grows beyond 15 variables. Although many new algorithms have been proposed to address this problem, it is not well understood which one is the best for which optimization scenario. In this work, we compare five state-of-the-art high-dimensional BO algorithms, with vanilla BO and CMA-ES on the 24 BBOB functions of the COCO environment at increasing dimensionality, ranging from 10 to 60 variables. Our results confirm the superiority of BO over CMA-ES for limited evaluation budgets and suggest that the most promising approach to improve BO is the use of trust regions. However, we also observe significant performance differences for different function landscapes and budget exploitation phases, indicating improvement potential, e.g., through hybridization of algorithmic components.

Authors 4

1. Maria Laura Santoni Aachen Leiden Institute of Advanced Computer Science Leiden Institute of Computer Science

   Università di Camerino · Leiden University · Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands

   Leiden Institute of Computer Science , , , The ,

   School of Science and Technology, University of Camerino, Italy

   Sorbonne Université, CNRS, LIP6, France

   Sorbonne Université, CNRS, LIP6, Paris, France,

   of and , of , , ,

2. Elena Raponi Aachen Leiden Institute of Advanced Computer Science Leiden Institute of Computer Science

   Università di Camerino · Leiden University · Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   Leiden Institute of Advanced Computer Science , Leiden University , Leiden , The Netherlands ,

   Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands

   Leiden Institute of Computer Science , , , The ,

   School of Science and Technology, University of Camerino, Italy

   Sorbonne Université, CNRS, LIP6, France

   Sorbonne Université, CNRS, LIP6, Paris, France,

   of and , of , , ,

3. Renato De Leone Aachen Leiden Institute of Advanced Computer Science Leiden Institute of Computer Science

   Università di Camerino · Leiden University · Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands

   Leiden Institute of Computer Science , , , The ,

   School of Science and Technology, University of Camerino, Italy

   Sorbonne Université, CNRS, LIP6, France

   Sorbonne Université, CNRS, LIP6, Paris, France,

   of and , of , , ,

4. Carola Doerr Aachen Leiden Institute of Advanced Computer Science Leiden Institute of Computer Science

   Università di Camerino · Leiden University · Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands

   Leiden Institute of Computer Science , , , The ,

   School of Science and Technology, University of Camerino, Italy

   Sorbonne Université, CNRS, LIP6, France

   Sorbonne Université, CNRS, LIP6, Paris, France,

   of and , of , , ,