Stochastic Gradient Descent for Constrained Optimization Based on Adaptive Relaxed Barrier Functions

IEEE Control Systems Letters, vol. 9, pp. 829–834

Abstract

This paper presents a novel stochastic gradient descent algorithm for constrained optimization. The proposed algorithm randomly samples constraints and components of the finite sum objective function and relies on a relaxed logarithmic barrier function that is appropriately adapted in each optimization iteration. For a strongly convex objective function and affine inequality constraints, step-size rules and barrier adaptation rules are established that guarantee asymptotic convergence with probability one. The theoretical results in the paper are complemented by numerical studies which highlight potential advantages of the proposed algorithm for optimization problems with a large number of constraints.

Authors 3

1. Naum Dimitrieski Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligent Control Systems, RWTH Aachen University, Aachen, Germany

2. Jing Cao Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligent Control Systems, RWTH Aachen University, Aachen, Germany

3. Christian Ebenbauer Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligent Control Systems, RWTH Aachen University, Aachen, Germany