Identifying Properties of Real-World Optimisation Problems through a Questionnaire

arXiv (Cornell University)

Abstract

Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design of future benchmark problems that more closely resemble those found in the real world. The results, while not representative as they are based on only 45 responses, indicate that many problems possess at least one of the following properties: they are constrained, deterministic, have only continuous variables, require substantial computation times for both the objectives and the constraints, or allow a limited number of evaluations. Properties like known optimal solutions and analytical gradients are rarely available, limiting the options in guiding the optimisation process. These are all important aspects to consider when designing realistic benchmark problems. At the same time, the design of realistic benchmarks is difficult, because objective functions are often reported to be black-box and many problem properties are unknown. To further improve the understanding of real-world problems, readers working on a real-world optimisation problem are encouraged to fill out the questionnaire: https://tinyurl.com/opt-survey

Authors 8

1. Koen van der Blom Aachen

   Leiden University

   Affiliation as printed

   Leiden University , Leiden , The Netherlands ,

2. Timo M. Deist

   Centrum Wiskunde & Informatica

   Affiliation as printed

   Centrum Wiskunde & Informatica , Amsterdam , The Nederlands ,

3. Vanessa Volz
   Affiliation as printed

   modl.ai , Copenhagen , Denmark

4. Mariapia Marchi
   Affiliation as printed

   Mariapia Marchi ESTECO SpA , Trieste , Italy,

5. Yusuke Nojima

   Osaka Prefecture University

   Affiliation as printed

   Osaka Prefecture University , Sakai , Osaka , Japan,

6. Boris Naujoks

   TH Köln - University of Applied Sciences · Jožef Stefan Institute

   Affiliation as printed

   TH Köln , Gummersbach , Germany,

   Tea Tušar Jožef Stefan Institute , Ljubljana , Slovenia,

7. Akira Oyama

   Japan Aerospace Exploration Agency

   Affiliation as printed

   Akira Oyama Japan Aerospace Exploration Agency , Sagamihara , Japan,

   Mariapia Marchi ESTECO SpA , Trieste , Italy,

8. Tea Tušar
   Affiliation as printed

   Mariapia Marchi ESTECO SpA , Trieste , Italy,