Fixed point geometry in chaotic neural networks

Physical Review Research, vol. 7

Abstract

Understanding the high-dimensional chaotic dynamics occurring in complex biological systems such as recurrent neural networks or ecosystems remains a conceptual challenge. For low-dimensional dynamics, fixed points provide the geometric scaffold of the dynamics. However, in high-dimensional systems, even the location of fixed points is unknown. Here, we analytically determine the number and distribution of fixed points for a canonical model of a recurrent neural network that exhibits high-dimensional chaos. This distribution reveals that fixed points and dynamics are confined to separate shells in state space. Furthermore, the distribution enables us to determine the eigenvalue spectra of the Jacobian at the fixed points, showing that each fixed point has a low-dimensional unstable manifold. Despite the radial separation of fixed points and dynamics, we find that the principal components of fixed points and dynamics align and that nearby fixed points act as partially attracting landmarks for the dynamics. Our results provide a detailed characterization of the fixed point geometry and its interplay with the dynamics, thereby paving the way towards a geometric understanding of high-dimensional chaos through their skeleton of unstable fixed points.

Authors 5

1. Jakob Stubenrauch corresponding Aachen Department of Physics Faculty 1

   Forschungszentrum Jülich · Humboldt-Universität zu Berlin · Bernstein Center for Computational Neuroscience Berlin · RWTH Aachen University

   Affiliation as printed

   Bernstein Center for Computational Neuroscience Berlin

   Jülich Research Centre

   Physics Department of Humboldt University Berlin

   RWTH Aachen University

   Bernstein Center for Computational Neuroscience Berlin, Philippstrasse 13, Haus 2, 10115 Berlin, Germany

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

   Physics Department of Humboldt University Berlin, Newtonstrasse 15, 12489 Berlin, Germany

2. Christian Keup Aachen Department of Physics Faculty 1

   Forschungszentrum Jülich · École Polytechnique Fédérale de Lausanne · RWTH Aachen University

   Affiliation as printed

   Jülich Research Centre

   RWTH Aachen University

   École Polytechnique Fédérale de Lausanne

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

3. Anno Christopher Kurth Aachen Department of Physics Faculty 1

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Jülich Research Centre

   RWTH Aachen University

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

4. Moritz Helias Aachen Department of Physics Faculty 1

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Jülich Research Centre

   RWTH Aachen University

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

5. Alexander van Meegen

   Harvard University · Forschungszentrum Jülich · University of Cologne

   Affiliation as printed

   Harvard University

   Jülich Research Centre

   University of Cologne

   Center for Brain Science, Harvard University, Cambridge, Massachusetts 02138, USA

   Institute of Zoology, University of Cologne, 50674 Cologne, Germany