A 1.1 μJ/Inference Binary Spiking Neural Network Accelerator for DVS Gesture Recognition

IEEE Transactions on Circuits and Systems I Regular Papers, vol. 73, pp. 1093–1106

Abstract

Dynamic vision sensors (DVS) are bioinspired sensors that can generate sparse data streams with low latency, low power consumption and high dynamic range. Spiking neural networks (SNNs), which are inspired by biological brains, are event-based models, and therefore they can be used to process the binary data streams produced by such sensors naturally. However, SNN accelerators usually require more memory and longer time for inference, due to the extra time dimension in SNNs. In this paper, an energy efficient binary spiking neural network (BSNN) accelerator for DVS gesture recognition is proposed with algorithm and hardware codesign. We integrate the binary neural network (BNN) training method into SNN to directly train a BSNN model, which significantly reduces memory consumption. A temporal pooling (TP) layer is further proposed to reduce the time steps in SNNs while maintaining competitive accuracy. The proposed BSNN accelerator can achieve high parallelism with high resource utilization, and the sparsity of input spikes is utilized to further reduce power consumption. The proposed BSNN model achieves an accuracy of 95.49% on IBM DVS Gesture dataset. The implementation results show that the BSNN accelerator can achieve 38.2k inference per second with$1.1~\mu $J/inference energy consumption and 216.9 TOPS/W energy efficiency.

Authors 5

1. Congyi Sun

   Nanjing University

   Affiliation as printed

   School of Electronic Science and Engineering, Nanjing University, Nanjing, China

2. Heng Zhang

   Nanjing University

   Affiliation as printed

   School of Electronic Science and Engineering, Nanjing University, Nanjing, China

3. Qinyu Chen Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands

4. Yuxiang Fu

   Nanjing University

   Affiliation as printed

   School of Integrated Circuits, Nanjing University, Nanjing, China

5. Li Li

   Nanjing University

   Affiliation as printed

   School of Electronic Science and Engineering, Nanjing University, Nanjing, China