NoisyDECOLLE: Robust Local Learning for SNNs on Neuromorphic Hardware

International Conference on Machine Learning and Applications, pp. 1535–1540

Abstract

Based on their biological archetype, spiking neural networks (SNNs) promise substantial energy savings at massively increased performance. However, best-performing SNNs in supervised learning scenarios often fall short of their potential efficiency gains as they rely on backpropagation, which entails issues like weight update locking until forward and backward passes are finished, and critical resource & computation requirements. These challenges can be tackled by drawing inspiration from biological synapses which rely largely on more local information to adapt. Local learning algorithms adopt this idea by employing local classifiers in each layer of an SNN that use only spatially and temporally local information to update synaptic weights. However, mapping these algorithms to neuromorphic systems to unleash their potential can be impaired by various kinds of noise. In this work, we review prior art to derive realistic noise scenarios on neuromorphic systems. Based on these results, we introduce NoisyDEColle, a framework for applying various noise models on locally learned SNNs using the DECOLLE network architecture. We show that both noise-aware training and additional regularization techniques allow NoisyDECOLLE to reach competitive performance, even under challenging conditions as for example posed by resistive devices. Using quantization-aware training, NoisyDECOLLE reaches 98.6% (94.1%) accuracy on N-MNIST (DVSGesture) with 3b (8b) weights ($> 4-10\mathrm{x}$memory and energy savings compared to 32b). To analyze our results, we provide the first time-driven implementation of spike activation maps, aiding the explainability of neuromorphic computing.

Authors 3

1. Tim Stadtmann Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University,Aachen,Germany

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University, Aachen, Germany

2. Benedikt Wahl Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University,Aachen,Germany

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University, Aachen, Germany

3. Tobias Gemmeke Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University,Aachen,Germany

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen University, Aachen, Germany