QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models

JuSER (Forschungszentrum Jülich)

Abstract

Structured State Space models (SSM) have recently emerged as a new class of deep learning models, particularly well-suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of Transformers, makes them attractive candidates for deployment on resource-constrained edge-computing devices. While recent works have explored the effect of quantization-aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in-memory computing (AIMC) chips. In this work, we demonstrate that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. We analyze the relation between model size and numerical precision, and show that QAT enhances robustness to analog noise and enables structural pruning. Finally, we integrate these techniques to deploy SSMs on a memristive analog in-memory computing substrate and highlight the resulting benefits in terms of computational efficiency.

Authors 5

1. Sebastian Siegel

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Peter Grünberg Institute (PGI-14) , Forschungszentrum Jülich GmbH and JARA , Jülich , Germany

2. Ming-Jay Yang

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Peter Grünberg Institute (PGI-14) , Forschungszentrum Jülich GmbH and JARA , Jülich , Germany

3. Younes Bouhadjar

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Peter Grünberg Institute (PGI-15) , Forschungszentrum Jülich GmbH and JARA , Jülich , Germany

4. Emre Neftci Aachen Faculty of Electrical Engineering

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Faculty of Electrical Engineering , RWTH Aachen University , Aachen , Germany

   Peter Grünberg Institute (PGI-15) , Forschungszentrum Jülich GmbH and JARA , Jülich , Germany

5. John Paul Strachan Aachen Faculty of Electrical Engineering

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Faculty of Electrical Engineering , RWTH Aachen University , Aachen , Germany

   Peter Grünberg Institute (PGI-14) , Forschungszentrum Jülich GmbH and JARA , Jülich , Germany