Large-Scale Benchmarking of Metaphor-Based Optimization Heuristics

Genetic and Evolutionary Computation Conference (GECCO), pp. 41–49

Abstract

The number of proposed iterative optimization heuristics is growing steadily, and with this growth, there have been many points of discussion within the wider community. One particular criticism that is raised towards many new algorithms is their focus on metaphors used to present the method, rather than emphasizing their potential algorithmic contributions. Several studies into popular metaphor-based algorithms have highlighted these problems, even showcasing algorithms that are functionally equivalent to older existing methods. Unfortunately, this detailed approach is not scalable to the whole set of metaphor-based algorithms. Because of this, we investigate ways in which benchmarking can shed light on these algorithms. To this end, we run a set of 294 algorithm implementations on the BBOB function suite. We investigate how the choice of the budget, the performance measure, or other aspects of experimental design impact the comparison of these algorithms. Our results emphasize why benchmarking is a key step in expanding our understanding of the algorithm space, and what challenges still need to be overcome to fully gauge the potential improvements to the state-of-the-art hiding behind the metaphors.

Authors 5

1. Diederick Vermetten Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, Netherlands

   LIACS - Leiden Institute of Advanced Computer Science [Leiden] (Niels Bohrweg 1 2333 CA Leiden - Netherlands)

2. Carola Doerr

   Centre National de la Recherche Scientifique · Sorbonne Université

   Affiliation as printed

   CNRS, Paris, France

   Sorbonne University, Paris, France

3. Hao Wang Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, Netherlands

   LIACS - Leiden Institute of Advanced Computer Science [Leiden] (Niels Bohrweg 1 2333 CA Leiden - Netherlands)

4. Anna V. Kononova Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, Netherlands

   LIACS - Leiden Institute of Advanced Computer Science [Leiden] (Niels Bohrweg 1 2333 CA Leiden - Netherlands)

5. Thomas Bäck Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, Netherlands

   LIACS - Leiden Institute of Advanced Computer Science [Leiden] (Niels Bohrweg 1 2333 CA Leiden - Netherlands)