Hardware-aware Few-shot Learning on a Memristor-based Small-world Architecture

Abstract

Learning from few examples (few-shot learning) is one of the hallmarks of mammalian intelligence. In the presented work, we demonstrate using simulations, on-chip few-shot learning on a recently-proposed Spiking Neural Network (SNN) hardware architecture, the Mosaic. The Mosaic exhibits a small-world property similar to that of a mammalian cortex, by virtue of its physical layout. Thanks to taking advantage of in-memory computing and routing along with local connectivity, the Mosaic is a highly efficient solution for routing information which is the main source of energy consumption in neural network accelerators, and specifically in neuromorphic hardware. We propose to meta-learn a small-world SNN resembling the Mosaic architecture for keyword spotting tasks using Model Agnostic Meta Learning (MAML) algorithm for adaptation on the edge and report the final accuracy on Spiking Heidelberg Digits dataset. Using simulations of hardware environment, we demonstrate 49.09 ± 8.17% accuracy on five unseen classes with 5-shot data and single gradient update. Furthermore, bumping it to 10 gradient steps we achieve an accuracy of 67.97 ± 1.99% on the same configuration. Our results show the applicability of MAML for analog substrates on the edge and highlight a few factors that impact the learning performance of such meta-learning models on neuromorphic substrates.

Authors 3

1. Yiğit Demirağ

   University of Zurich · ETH Zurich · Institute of Neuroinformatics

   Affiliation as printed

   University of Zürich and ETH Zürich,Institute of Neuroinformatics,Zürich,Switzerland

2. Emre Neftci Aachen Peter Grünberg Institute (PGI 10)

   RWTH Aachen University

   Affiliation as printed

   Forschungszentrum Jülich and RWTH Aachen,Peter Grünberg Institute,Aachen,Germany

3. Melika Payvand

   University of Zurich · ETH Zurich · Institute of Neuroinformatics

   Affiliation as printed

   University of Zürich and ETH Zürich,Institute of Neuroinformatics,Zürich,Switzerland