Routing brain traffic through the von Neumann bottleneck: Efficient cache usage in spiking neural network simulation code on general purpose computers

Parallel Computing, vol. 113, pp. 102952

Abstract

Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed random graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. The activity in neuronal networks is also sparse. Each neuron occasionally transmits a brief signal, called spike, via its outgoing synapses to the corresponding target neurons. In distributed computing these targets are scattered across thousands of parallel processes. The spatial and temporal sparsity represents an inherent bottleneck for simulations on conventional computers: irregular memory-access patterns cause poor cache utilization. Using an established neuronal network simulation code as a reference implementation, we investigate how common techniques to recover cache performance such as software-induced prefetching and software pipelining can benefit a real-world application. The algorithmic changes reduce simulation time by up to 50%. The study exemplifies that many-core systems assigned with an intrinsically parallel computational problem can alleviate the von Neumann bottleneck of conventional computer architectures.

Authors 6

1. Jari Pronold corresponding Aachen

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

2. Jakob Jordan

   University of Bern

   Affiliation as printed

   Department of Physiology, University of Bern, Bern, Switzerland

3. Brian J. N. Wylie

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre, Jülich Research Centre, Jülich, Germany

4. Itaru Kitayama

   RIKEN Center for Computational Science

   Affiliation as printed

   RIKEN Center for Computational Science, Kobe, Japan

5. Markus Diesmann Aachen Department of Physics Faculty 1 Department of Psychiatry, Psychotherapy and Psychosomatics Medical Faculty

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Department of Physics, Faculty 1, RWTH Aachen University, Aachen, Germany

   Department of Psychiatry, Psychotherapy and Psychosomatics, Medical Faculty, RWTH Aachen University, Aachen, Germany

   Institute of Neuroscience and Medicine (INM-6) and Institute for Advanced Simulation (IAS-6) and JARA-Institute Brain Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany

6. Susanne Kunkel corresponding

   Norwegian University of Life Sciences

   Affiliation as printed

   Faculty of Science and Technology, Norwegian University of Life Sciences, Ås, Norway