A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations

Frontiers in Neuroinformatics, vol. 16, pp. 837549

Abstract

Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale models that study interactions between multiple brain areas with intricate connectivity and investigate phenomena on long time scales such as system-level learning require progress in simulation speed. The corresponding development of state-of-the-art simulation engines relies on information provided by benchmark simulations which assess the time-to-solution for scientifically relevant, complementary network models using various combinations of hardware and software revisions. However, maintaining comparability of benchmark results is difficult due to a lack of standardized specifications for measuring the scaling performance of simulators on high-performance computing (HPC) systems. Motivated by the challenging complexity of benchmarking, we define a generic workflow that decomposes the endeavor into unique segments consisting of separate modules. As a reference implementation for the conceptual workflow, we develop beNNch: an open-source software framework for the configuration, execution, and analysis of benchmarks for neuronal network simulations. The framework records benchmarking data and metadata in a unified way to foster reproducibility. For illustration, we measure the performance of various versions of the NEST simulator across network models with different levels of complexity on a contemporary HPC system, demonstrating how performance bottlenecks can be identified, ultimately guiding the development toward more efficient simulation technology.

Authors 12

1. Jasper Albers corresponding Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

2. Jari Pronold Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

3. Anno Christopher Kurth Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

4. Stine Brekke Vennemo

   Norwegian University of Life Sciences

   Affiliation as printed

   Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway

5. Kaveh Haghighi Mood

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre (JSC), Jülich Research Centre, Jülich, Germany

6. Alexander Patronis

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre (JSC), Jülich Research Centre, Jülich, Germany

7. D. Terhorst

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

8. Jakob Jordan

   University of Bern

   Affiliation as printed

   Department of Physiology, University of Bern, Bern, Switzerland

9. Susanne Kunkel

   Norwegian University of Life Sciences

   Affiliation as printed

   Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway

10. Tom Tetzlaff

    Forschungszentrum Jülich · Jülich Aachen Research Alliance

    Affiliation as printed

    Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

11. Markus Diesmann Aachen Department of Physics Faculty 1 Department of Psychiatry, Psychotherapy and Psychosomatics School of Medicine

    Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

    Affiliation as printed

    Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

    Department of Psychiatry, Psychotherapy and Psychosomatics, School of Medicine, RWTH Aachen University, Aachen, Germany

    Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

12. Johanna Senk

    Forschungszentrum Jülich · Jülich Aachen Research Alliance

    Affiliation as printed

    Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany