An Adaptive Accelerated Derivative-Free Optimization Algorithm Based on Noncommutative Maps

IEEE Transactions on Automatic Control, vol. 71, pp. 3559–3574

Abstract

In this article, an adaptive accelerated derivative-free optimization algorithm is developed. A composition of noncommutative maps based on objective function evaluations is used to approximate an accelerated gradient descent algorithm with a momentum term. An adaptive step-size rule and an adaptive momentum term are introduced to improve the algorithm's performance in terms of convergence speed and steady-state accuracy. Semi-global asymptotic stability of the proposed algorithm is proved for a class of convex objective functions under suitable assumptions. Simulation results are shown and compared to other derivative-free optimization algorithms.

Authors 3

1. Xin Huo

   Harbin Institute of Technology

   Affiliation as printed

   Control and Simulation Center, Harbin Institute of Technology, Harbin, China

2. Christian Ebenbauer Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligent Control Systems, RWTH Aachen University, Aachen, Germany

3. Kemao Ma

   Harbin Institute of Technology

   Affiliation as printed

   Control and Simulation Center, Harbin Institute of Technology, Harbin, China