Optimization Algorithm Synthesis Based on Integral Quadratic Constraints: A Tutorial

Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes, pp. 2995–3002

Abstract

We expose in a tutorial fashion the mechanisms which underlie the synthesis of optimization algorithms based on dynamic integral quadratic constraints. We reveal how these tools from robust control allow to design accelerated gradient descent algorithms with optimal guaranteed convergence rates by solving small-sized convex semi-definite programs. It is shown that this extends to the design of extremum controllers, with the goal to regulate the output of a general linear closed-loop system to the minimum of an objective function. Numerical experiments illustrate that we can not only recover gradient decent and the triple momentum variant of Nesterov's accelerated first order algorithm, but also automatically syn-thesize optimal algorithms even if the gradient information is passed through non-trivial dynamics, such as time-delays.

Authors 3

1. Carsten W. Scherer

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Department of Mathematics,Germany

   Department of Mathematics, University of Stuttgart, Germany

2. Christian Ebenbauer Aachen Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Intelligent Control Systems,Germany

   Intelligent Control Systems, RWTH Aachen University, Germany

3. Tobias Holicki

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Department of Mathematics,Germany

   Department of Mathematics, University of Stuttgart, Germany