Neural network training under semidefinite constraints

Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes, pp. 2731–2736

Abstract

This paper is concerned with the training of neural networks (NNs) under semidefinite constraints, which allows for NN training with robustness and stability guarantees. In particular, we focus on Lipschitz bounds for NNs. Exploiting the banded structure of the underlying matrix constraint, we set up an efficient and scalable training scheme for NN training problems of this kind based on interior point methods. Our implementation allows to enforce Lipschitz constraints in the training of large-scale deep NNs such as Wasserstein generative adversarial networks (WGANs) via semidefinite constraints. In numerical examples, we show the superiority of our method and its applicability to WGAN training.

Authors 5

1. Patricia Pauli

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Institute for Systems Theory and Automatic Control,Stuttgart,Germany,70569

2. Niklas Funcke

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Institute for Systems Theory and Automatic Control,Stuttgart,Germany,70569

3. Dennis Gramlich Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen,Chair of Intelligent Control Systems,Aachen,Germany,52074

4. Mohamed Amine Msalmi

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Institute for Systems Theory and Automatic Control,Stuttgart,Germany,70569

5. Frank Allgöwer

   University of Stuttgart

   Affiliation as printed

   University of Stuttgart,Institute for Systems Theory and Automatic Control,Stuttgart,Germany,70569