Parallel Golomb-Rice Decoder with 8-bit Unary Decoding for Weight Compression in TinyML Applications

Euromicro Conference on Digital System Design (DSD), pp. 227–234

Abstract

Due to the recent advances in AI, the requirement for Artificial Intelligence (AI) has increased exponentially in the domain of Internet of Things (IoT). Running Deep Neural Networks (DNNs) on edge devices gives the advantage of privacy, security, and lower latency. It is challenging to deploy them on embedded devices with constrained hardware resources since a lot of compute and memory resources are required. Memory access contributes to the majority of the energy requirements on edge devices. Although data compression plays a critical role in reducing storage and memory bandwidth requirements, most of the hardware decoders are inefficient in terms of power, area, and throughput. In this work, a hardware Parallel Golomb-Rice decoder is presented that can decode 8-bits of unary encoded data every cycle. The design has been integrated with a Neural Network (NN) accelerator and experimented with state-of-the-art benchmark models. Lossless compression is performed with an offline Golomb-Rice encoder. It encodes the weights of each layer with an optimum Golomb-Rice parameter. Applied to the benchmarks Anomaly Detection, Image Classification and Visual Wake Words the memory access during inference is reduced by 26.8%, 6.62% and 5.54% respectively. The decoder dissipates 0.4216 mW of power and delivers an average throughput of 860 MBps. The design has been synthesised with 40 nm technology and compared with state-of-the-art works.

Authors 5

1. Mounika Vaddeboina

   Infineon Technologies (Germany) · Technical University of Munich

   Affiliation as printed

   Infineon Technologies AG,Germany

   Infineon Technologies AG, Germany

   Technische Universitat Munchen, Germany

2. Endri Kaja

   Infineon Technologies (Germany) · Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau

   Affiliation as printed

   Infineon Technologies AG,Germany

   Infineon Technologies AG, Germany

   Rheinland-Pfalzische Technische Universitat Kaiserslautern-Landau, Germany

3. Alper Yilmayer Aachen

   Infineon Technologies (Germany) · RWTH Aachen University

   Affiliation as printed

   Infineon Technologies AG,Germany

   Infineon Technologies AG, Germany

   Rheinisch- Westfalische Technische Hochschule Aachen, Germany

4. Sebastian Siegfried Prebeck

   Infineon Technologies (Germany) · Technical University of Munich

   Affiliation as printed

   Infineon Technologies AG,Germany

   Infineon Technologies AG, Germany

   Technische Universitat Munchen, Germany

5. Wolfgang Ecker

   Infineon Technologies (Germany) · Technical University of Munich

   Affiliation as printed

   Infineon Technologies AG,Germany

   Infineon Technologies AG, Germany

   Technische Universitat Munchen, Germany