Quantifying Individual and Joint Module Impact in Modular Optimization Frameworks

arXiv (Cornell University)

Abstract

This study explores the influence of modules on the performance of modular optimization frameworks for continuous single-objective black-box optimization. There is an extensive variety of modules to choose from when designing algorithm variants, however, there is a rather limited understanding of how each module individually influences the algorithm performance and how the modules interact with each other when combined. We use the functional ANOVA (f-ANOVA) framework to quantify the influence of individual modules and module combinations for two algorithms, the modular Covariance Matrix Adaptation (modCMA) and the modular Differential Evolution (modDE). We analyze the performance data from 324 modCMA and 576 modDE variants on the BBOB benchmark collection, for two problem dimensions, and three computational budgets. Noteworthy findings include the identification of important modules that strongly influence the performance of modCMA, such as the~\textit{weights\ option} and~\textit{mirrored} modules for low dimensional problems, and the~\textit{base\ sampler} for high dimensional problems. The large individual influence of the~\textit{lpsr} module makes it very important for the performance of modDE, regardless of the problem dimensionality and the computational budget. When comparing modCMA and modDE, modDE undergoes a shift from individual modules being more influential, to module combinations being more influential, while modCMA follows the opposite pattern, with an increase in problem dimensionality and computational budget.

Authors 5

1. Ana Nikolikj

   Jožef Stefan Institute · Jožef Stefan International Postgraduate School

   Affiliation as printed

   IJS - Jozef Stefan Institute [Ljubljana] (Jamova cesta 39, 1000 Ljubljana Slovenia - Slovenia)

2. Ana Kostovska

   Jožef Stefan Institute · Jožef Stefan International Postgraduate School

   Affiliation as printed

   IJS - Jozef Stefan Institute [Ljubljana] (Jamova cesta 39, 1000 Ljubljana Slovenia - Slovenia)

3. Diederick Vermetten Aachen

   Leiden University · Jožef Stefan Institute · Sorbonne Université · Jožef Stefan International Postgraduate School

   Affiliation as printed

   Computer Systems Department , Jožef Stefan Institute , Ljubljana , Slovenia

   Sorbonne Université ,

   LIACS - Leiden Institute of Advanced Computer Science [Leiden] ( Niels Bohrweg 1 2333 CA Leiden - Netherlands)

4. Carola Doerr

   Jožef Stefan Institute · Jožef Stefan International Postgraduate School

   Affiliation as printed

   Computer Systems Department , Jožef Stefan Institute , Ljubljana , Slovenia

   RO - Recherche Opérationnelle (France)

5. Tome Eftimov

   Jožef Stefan Institute · Jožef Stefan International Postgraduate School

   Affiliation as printed

   Computer Systems Department , Jožef Stefan Institute , Ljubljana , Slovenia

   IJS - Jozef Stefan Institute [Ljubljana] (Jamova cesta 39, 1000 Ljubljana Slovenia - Slovenia)