Path classification by stochastic linear recurrent neural networks

Advances in Continuous and Discrete Models, vol. 2022

Abstract

Abstract We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with the identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and support these observations with numerical experiments on both synthetic and real data. We also show a trade-off phenomenon between accuracy and robustness.

Authors 4

1. Youness Boutaib corresponding Aachen Chair for Mathematics of Information Processing

   RWTH Aachen University

   Affiliation as printed

   Chair for Mathematics of Information Processing, RWTH Aachen University, Pontdriesch 10, 52062, Aachen, Germany

   Chair for Mathematics of Information Processing, RWTH Aachen University, Aachen, Germany

2. Wiebke Bartolomaeus Aachen Chair for Mathematics of Information Processing

   RWTH Aachen University

   Affiliation as printed

   Chair for Mathematics of Information Processing, RWTH Aachen University, Pontdriesch 10, 52062, Aachen, Germany

   Chair for Mathematics of Information Processing, RWTH Aachen University, Aachen, Germany

3. Sandra Nestler Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute “Brain Structure-Function Relationships” (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

4. Holger Rauhut Aachen Chair for Mathematics of Information Processing

   RWTH Aachen University

   Affiliation as printed

   Chair for Mathematics of Information Processing, RWTH Aachen University, Pontdriesch 10, 52062, Aachen, Germany

   Chair for Mathematics of Information Processing, RWTH Aachen University, Aachen, Germany