Neural network training under semidefinite constraints

RWTH Publications (RWTH Aachen)

Abstract

This paper is concerned with the training of neural networks (NNs) under semidefinite constraints, which allows for NN training with robustness and stability guarantees. In particular, we focus on Lipschitz bounds for NNs. Exploiting the banded structure of the underlying matrix constraint, we set up an efficient and scalable training scheme for NN training problems of this kind based on interior point methods. Our implementation allows to enforce Lipschitz constraints in the training of large-scale deep NNs such as Wasserstein generative adversarial networks (WGANs) via semidefinite constraints. In numerical examples, we show the superiority of our method and its applicability to WGAN training.

Authors 5

1. Patricia Pauli

   University of Stuttgart

   Affiliation as printed

   Institute for Systems Theory and Au- tomatic Control , University of Stuttgart , 70569 Stuttgart , Germany

2. Niklas Funcke

   University of Stuttgart

   Affiliation as printed

   Institute for Systems Theory and Au- tomatic Control , University of Stuttgart , 70569 Stuttgart , Germany

3. Dennis Gramlich Aachen Chair of Intelligent Control Systems

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligent Control Systems , RWTH Aachen , 52074 Aachen , Germany

4. Mohamed Amine Msalmi

   University of Stuttgart

   Affiliation as printed

   Institute for Systems Theory and Au- tomatic Control , University of Stuttgart , 70569 Stuttgart , Germany

5. Frank Allgöwer