Unfolding recurrence by Green's functions for optimized reservoir computing

arXiv (Cornell University), vol. 33, pp. 17380–17390

Abstract

Cortical networks are strongly recurrent, and neurons have intrinsic temporal dynamics. This sets them apart from deep feed-forward networks. Despite the tremendous progress in the application of feed-forward networks and their theoretical understanding, it remains unclear how the interplay of recurrence and non-linearities in recurrent cortical networks contributes to their function. The purpose of this work is to present a solvable recurrent network model that links to feed forward networks. By perturbative methods we transform the time-continuous, recurrent dynamics into an effective feed-forward structure of linear and non-linear temporal kernels. The resulting analytical expressions allow us to build optimal time-series classifiers from random reservoir networks. Firstly, this allows us to optimize not only the readout vectors, but also the input projection, demonstrating a strong potential performance gain. Secondly, the analysis exposes how the second order stimulus statistics is a crucial element that interacts with the non-linearity of the dynamics and boosts performance.

Authors 6

1. Sandra Nestler

   Forschungszentrum Jülich

   Affiliation as printed

   Forschungszentrum Julich

2. Christian Keup Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) , Jülich Research Centre , Jülich , Germany

   RWTH Aachen University , Aachen , Germany

   Forschungszentrum Julich

3. David Dahmen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) , Jülich Research Centre , Jülich , Germany

   Forschungszentrum Julich

4. Matthieu Gilson

   Universitat Pompeu Fabra · Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) , Jülich Research Centre , Jülich , Germany

   Universitat Pompeu Fabra , Barcelona , Spain

   Forschungszentrum Julich

5. Holger Rauhut Aachen Mathematics of Information Processing

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Mathematics of Information Processing , RWTH Aachen University , Aachen , Germany

   Forschungszentrum Julich

6. Moritz Helias Aachen Department of Physics

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Department of Physics , RWTH Aachen University , Aachen , Germany

   Institute of Neuroscience and Medicine (INM-6) , Jülich Research Centre , Jülich , Germany

   RWTH Aachen University