CIMFlow: Modelling Dataflow in Cross-Layer Compute-in-Memory Deep Learning Accelerators

ACM Transactions on Embedded Computing Systems, vol. 24, pp. 1–22

Abstract

Traditional Deep Learning Accelerators (DLAs) rely on off-chip memory to store large weight tensors, leading to high bandwidth demands and energy consumption. Compute-in-Memory (CIM) accelerators mitigate this by integrating high-density, non-volatile memory arrays, enabling a fully weight-stationary (FWS) dataflow. Multi-core CIM systems further enhance efficiency with cross-layer inference, where intermediate tensors stay on-chip, and cores operate in a pipeline. Despite diverse architecture proposals, no existing tool models the dataflow, memory access patterns and timing behaviour of multi-core CIM accelerators. We introduce CIMFlow, a modelling framework for cross-layer CIM architectures. Our flexible Hardware Architecture Model includes CIM and digital cores and leverages the buffets storage idiom for distributed token-based flow control. An Array-OL-based Workload Model captures CNNs’ multidimensional dependencies and applies hardware-aware transformations. These models are transformed into a timed cyclo-static dataflow graph for simulation. CIMFlow delivers latency, energy and traces for core and buffer utilisation. Our case studies on state-of-the-art CNNs show that cross-layer inference reduces latency by up to 52×. We also reveal that neglecting memory access delays results in throughput overestimations of up to 308%. To our knowledge, CIMFlow is the first tool focused on FWS cross-layer execution in CIM architectures that explicitly models data movement costs. It serves as a powerful cost model for design space exploration in next-generation CIM accelerators.

Authors 6

1. José Cubero-Cascante Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen, Germany

2. Lenka Schneider Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen Germany

3. Rebecca Pelke Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen Germany

4. Arunkumar M. Vaidyanathan Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen Germany

5. Rainer Leupers Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen Germany

6. Jan Moritz Joseph Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen Germany