A multiscale consensus-based algorithm for multilevel optimization

Mathematical Models and Methods in Applied Sciences, vol. 35, pp. 2207–2243

Abstract

In this paper, a novel multiscale consensus-based optimization (CBO) algorithm for solving bi- and tri-level optimization problems is introduced. Existing CBO techniques are generalized by the proposed method through the employment of multiple interacting populations of particles, each of which is used to optimize one level of the problem. These particle populations are evolved through multiscale-in-time dynamics, which are formulated as a singularly perturbed system of stochastic differential equations. Theoretical convergence analysis for the multiscale CBO model to an averaged effective dynamics as the time-scale separation parameter approaches zero is provided. The resulting algorithm is presented for both bi-level and tri-level optimization problems. The effectiveness of the approach in tackling complex multilevel optimization tasks is demonstrated through numerical experiments on various benchmark functions. Additionally, it is shown that the proposed method performs well on min–max optimization problems, comparing favorably with existing CBO algorithms for saddle point problems.

Authors 4

1. Michaël Herty Aachen Institut für Geometrie und Praktische Mathematik

   RWTH Aachen University · University of Pretoria

   Affiliation as printed

   Extraordinary Professor, Department of Mathematics and Applied Mathematics, University of Pretoria, Private Bag X20, Hatfield 0028, South Africa

   Institut für Geometrie und Praktische Mathematik, RWTH Aachen University, Templergraben 55, 52062 Aachen, Germany

2. Yuyang Huang

   Imperial College London

   Affiliation as printed

   Department of Mathematics, Imperial College London, South Kensington Campus, SW72AZ London, UK

3. Dante Kalise

   Imperial College London

   Affiliation as printed

   Department of Mathematics, Imperial College London, South Kensington Campus, SW72AZ London, UK

4. Hicham Kouhkouh

   University of Graz · Nawi Graz

   Affiliation as printed

   Department of Mathematics and Scientific Computing, NAWI, University of Graz, Heinrichstraße 36, 8010 Graz, Austria