Transient Chaotic Dimensionality Expansion by Recurrent Networks

Physical Review X, vol. 11

Abstract

Neurons in the brain communicate with spikes, which are discrete events in time and value.Functional network models often employ rate units that are continuously coupled by analog signals.Is there a qualitative difference implied by these two forms of signaling?We develop a unified mean-field theory for large random networks to show that first-and second-order statistics in rate and binary networks are in fact identical if rate neurons receive the right amount of noise.Their response to presented stimuli, however, can be radically different.We quantify these differences by studying how nearby state trajectories evolve over time, asking to what extent the dynamics is chaotic.Chaos in the two models is found to be qualitatively different.In binary networks, we find a network-size-dependent transition to chaos and a chaotic submanifold whose dimensionality expands stereotypically with time, while rate networks with matched statistics are nonchaotic.Dimensionality expansion in chaotic binary networks aids classification in reservoir computing and optimal performance is reached within about a single activation per neuron; a fast mechanism for computation that we demonstrate also in spiking networks.A generalization of this mechanism extends to rate networks in their respective chaotic regimes.

Authors 4

1. Christian Keup Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA Institut Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

2. Tobias Kühn Aachen

   Centre National de la Recherche Scientifique · Forschungszentrum Jülich · École Normale Supérieure - PSL · Laboratoire Matière et Systèmes Complexes · Laboratoire de Physique de l'Ecole Normale Supérieure · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA Institut Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   Laboratoire de Physique de l’ENS, Laboratoire MSC de l’Université de Paris, CNRS, Paris, France

   RWTH Aachen University, Aachen, Germany

   Laboratoire de Physique de l'ENS, Laboratoire MSC de l'Université de Paris, CNRS, Paris, France

3. David Dahmen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA Institut Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

4. Moritz Helias Aachen Department of Physics Faculty 1

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA Institut Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany