QS4D: Quantization‐Aware Training for Efficient Hardware Deployment of Structured State‐Space Sequential Models

Advanced Intelligent Systems, vol. 8

Abstract

Structured state space models (SSM) have recently emerged as a new class of deep learning models, particularly well‐suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of transformers, makes them attractive candidates for deployment on resource‐constrained edge‐computing devices. While recent works have explored the effect of quantization‐aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in‐memory computing (AIMC) chips. In this work, it is demonstrated that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. The relation between model size and numerical precision is analyzed, and it is shown that QAT enhances robustness to analog noise and enables structural pruning. Finally, these techniques are integrated to deploy SSMs on a memristive AIMC substrate and highlight the resulting benefits in terms of computational efficiency.

Authors 6

1. Sebastian Siegel corresponding

   Forschungszentrum Jülich

   Affiliation as printed

   Peter‐Grünberg‐Institute (PGI‐14) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany

2. Ming-Jay Yang

   Forschungszentrum Jülich

   Affiliation as printed

   Peter‐Grünberg‐Institute (PGI‐14) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany

3. Younes Bouhadjar

   Forschungszentrum Jülich

   Affiliation as printed

   Peter‐Grünberg‐Institute (PGI‐15) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany

4. Maxime Fabre

   Forschungszentrum Jülich · University of Groningen

   Affiliation as printed

   Groningen Cognitive Systems and Materials Center (CogniGron) University of Groningen 9700 AB Groningen The Netherlands

   Peter‐Grünberg‐Institute (PGI‐15) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany

5. Emre Neftci Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University · Forschungszentrum Jülich

   Affiliation as printed

   Faculty of Electrical Engineering and Information Technology RWTH Aachen University Templergraben 55 52062 Aachen Germany

   Peter‐Grünberg‐Institute (PGI‐15) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany

6. John Paul Strachan Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University · Forschungszentrum Jülich

   Affiliation as printed

   Faculty of Electrical Engineering and Information Technology RWTH Aachen University Templergraben 55 52062 Aachen Germany

   Peter‐Grünberg‐Institute (PGI‐14) Forschungszentrum Jülich GmbH Wilhelm‐Johnen‐Straße 52428 Jülich Germany