Learning adaptive differential evolution algorithm from optimization experiences by policy gradient

arXiv (Cornell University)

Abstract

Differential evolution is one of the most prestigious population-based stochastic optimization algorithm for black-box problems. The performance of a differential evolution algorithm depends highly on its mutation and crossover strategy and associated control parameters. However, the determination process for the most suitable parameter setting is troublesome and time-consuming. Adaptive control parameter methods that can adapt to problem landscape and optimization environment are more preferable than fixed parameter settings. This paper proposes a novel adaptive parameter control approach based on learning from the optimization experiences over a set of problems. In the approach, the parameter control is modeled as a finite-horizon Markov decision process. A reinforcement learning algorithm, named policy gradient, is applied to learn an agent (i.e. parameter controller) that can provide the control parameters of a proposed differential evolution adaptively during the search procedure. The differential evolution algorithm based on the learned agent is compared against nine well-known evolutionary algorithms on the CEC'13 and CEC'17 test suites. Experimental results show that the proposed algorithm performs competitively against these compared algorithms on the test suites.

Authors 4

1. Jianyong Sun Aachen Leiden Institute of Advanced Computer Science

   Leiden University · Xi'an Jiaotong University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science , Leiden University , Leiden , The Netherlands

   School of Mathematics and Statistics , Xi'an Jiaotong University , Xi'an , China , 710049

   Xi’an Jiaotong University#TAB#

2. Xin Liu

   Xi'an Jiaotong University

   Affiliation as printed

   School of Mathematics and Statistics , Xi'an Jiaotong University , Xi'an , China , 710049

   Xi’an Jiaotong University#TAB#

3. Thomas Bäck

   Xi'an Jiaotong University

   Affiliation as printed

   School of Mathematics and Statistics , Xi'an Jiaotong University , Xi'an , China , 710049

   Xi’an Jiaotong University#TAB#

4. Zongben Xu Aachen

   Leiden University · Xi'an Jiaotong University

   Affiliation as printed

   School of Mathematics and Statistics , Xi'an Jiaotong University , Xi'an , China , 710049

   Leiden University