Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure

Frontiers in Neuroinformatics, vol. 16, pp. 884180

Abstract

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

Authors 22

1. Benedikt Feldotto corresponding

   Technical University of Munich

   Affiliation as printed

   Robotics, Artificial Intelligence and Real-Time Systems, Faculty of Informatics, Technical University of Munich, Munich, Germany

2. Jochen Martin Eppler

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · Jülich Supercomputing Centre

   Affiliation as printed

   Simulation and Data Lab Neuroscience, Jülich Supercomputing Centre (JSC), Institute for Advanced Simulation, JARA, Forschungszentrum Jülich GmbH, Jülich, Germany

3. Cristian Jimenez-Romero

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · Jülich Supercomputing Centre

   Affiliation as printed

   Simulation and Data Lab Neuroscience, Jülich Supercomputing Centre (JSC), Institute for Advanced Simulation, JARA, Forschungszentrum Jülich GmbH, Jülich, Germany

4. christopher bignamini

   ETH Zurich · CSCS - Swiss National Supercomputing Centre

   Affiliation as printed

   Swiss National Supercomputing Centre (CSCS), ETH Zurich, Lugano, Switzerland

5. Carlos Enrique Gutierrez

   Okinawa Institute of Science and Technology Graduate University

   Affiliation as printed

   Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan

6. Ugo Albanese

   Scuola Superiore Sant'Anna

   Affiliation as printed

   Department of Excellence in Robotics and AI, The BioRobotics Institute, Scuola Superiore Sant'Anna, Pontedera, Italy

7. Eloy Retamino

   Universidad de Granada

   Affiliation as printed

   Department of Computer Architecture and Technology, Research Centre for Information and Communication Technologies, University of Granada, Granada, Spain

8. Viktor V. Vorobev

   Technical University of Munich

   Affiliation as printed

   Robotics, Artificial Intelligence and Real-Time Systems, Faculty of Informatics, Technical University of Munich, Munich, Germany

9. Vahid Zolfaghari

   Technical University of Munich

   Affiliation as printed

   Robotics, Artificial Intelligence and Real-Time Systems, Faculty of Informatics, Technical University of Munich, Munich, Germany

10. Alex Upton

    ETH Zurich · CSCS - Swiss National Supercomputing Centre

    Affiliation as printed

    Swiss National Supercomputing Centre (CSCS), ETH Zurich, Lugano, Switzerland

11. Zhe Sun

    RIKEN · RIKEN Center for Advanced Photonics

    Affiliation as printed

    Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Wako, Japan

    Image Processing Research Team, Center for Advanced Photonics, RIKEN, Wako, Japan

12. Hiroshi YAMAURA

    University of Electro-Communications

    Affiliation as printed

    Graduate School of Informatics and Engineering, The University of Electro-Communications, Tokyo, Japan

13. Morteza Heidarinejad

    RIKEN

    Affiliation as printed

    Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Wako, Japan

14. Wouter Klijn

    Forschungszentrum Jülich · Jülich Aachen Research Alliance · Jülich Supercomputing Centre

    Affiliation as printed

    Simulation and Data Lab Neuroscience, Jülich Supercomputing Centre (JSC), Institute for Advanced Simulation, JARA, Forschungszentrum Jülich GmbH, Jülich, Germany

15. Abigail Morrison Aachen Computer Science 3 - Software Engineering

    Forschungszentrum Jülich · Jülich Aachen Research Alliance · Jülich Supercomputing Centre · RWTH Aachen University

    Affiliation as printed

    Computer Science 3-Software Engineering, RWTH Aachen University, Aachen, Germany

    Jülich Research Centre, Institute of Neuroscience and Medicine (INM-6), Institute for Advanced Simulation (IAS-6), JARA BRAIN Institute I, Jülich, Germany

    Simulation and Data Lab Neuroscience, Jülich Supercomputing Centre (JSC), Institute for Advanced Simulation, JARA, Forschungszentrum Jülich GmbH, Jülich, Germany

16. Felipe A. Cruz

    ETH Zurich · CSCS - Swiss National Supercomputing Centre

    Affiliation as printed

    Swiss National Supercomputing Centre (CSCS), ETH Zurich, Lugano, Switzerland

17. Colin J. McMurtrie

    ETH Zurich · CSCS - Swiss National Supercomputing Centre

    Affiliation as printed

    Swiss National Supercomputing Centre (CSCS), ETH Zurich, Lugano, Switzerland

18. Alois Knoll

    Technical University of Munich

    Affiliation as printed

    Robotics, Artificial Intelligence and Real-Time Systems, Faculty of Informatics, Technical University of Munich, Munich, Germany

19. Jun Igarashi

    RIKEN · RIKEN Center for Computational Science

    Affiliation as printed

    Center for Computational Science, RIKEN, Kobe, Japan

    Computational Engineering Applications Unit, Head Office for Information Systems and Cybersecurity, RIKEN, Wako, Japan

20. Tadashi Yamazaki

    University of Electro-Communications

    Affiliation as printed

    Graduate School of Informatics and Engineering, The University of Electro-Communications, Tokyo, Japan

21. Kenji Doya

    Okinawa Institute of Science and Technology Graduate University

    Affiliation as printed

    Neural Computation Unit, Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan

22. Fabrice Olivier Morin

    Technical University of Munich

    Affiliation as printed

    Robotics, Artificial Intelligence and Real-Time Systems, Faculty of Informatics, Technical University of Munich, Munich, Germany