Controllability of continuous networks and a kernel-based learning approximation

arXiv (Cornell University)

Abstract

Residual deep neural networks are formulated as interacting particle systems leading to a description through neural differential equations, and, in the case of large input data, through mean-field neural networks. The mean-field description allows also the recast of the training processes as a controllability problem for the solution to the mean-field dynamics. We show theoretical results on the controllability of the linear microscopic and mean-field dynamics through the Hilbert Uniqueness Method and propose a computational approach based on kernel learning methods to solve numerically, and efficiently, the training problem. Further aspects of the structural properties of the mean-field equation will be reviewed.

Authors 3

1. Michaël Herty Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen

2. Chiara Segala Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen

3. Giuseppe Visconti