RobustDiCE: Robust and Distributed CNN Inference at the Edge

Asia and South Pacific Design Automation Conference (ASP-DAC), pp. 26–31

Abstract

Prevalent large CNN models pose a significant challenge in terms of computing resources for resource-constrained devices at the Edge. Distributing the computations and coefficients over multiple edge devices collaboratively has been well studied but these works generally do not consider the presence of device failures (e.g., due to temporary connectivity issues, overload, discharged battery, etc. of edge devices). Such unpredictable failures can compromise the reliability of edge devices, inhibiting the proper execution of distributed CNN inference. In this paper, we present a novel partitioning method, called RobustDiCE, for robust distribution and inference of CNN models over multiple edge devices. Our method can tolerate intermittent and permanent device failures in a distributed system at the Edge, offering a tunable trade-off between robustness (i.e., retaining model accuracy after failures) and resource utilization. We evaluate RobustDiCE using the ImageNet-1K dataset on several representative CNN models under various device failure scenarios and compare it with several state-of-the-art partitioning methods as well as an optimal robustness approach (i.e., full neuron replication). In addition, we demonstrate RobustDiCE’s advantages in terms of memory usage and energy consumption per device, and system throughput for various system set-ups with different device counts.

Authors 4

1. Xiaotian Guo Aachen Univ. of Amsterdam

   Leiden University · University of Amsterdam

   Affiliation as printed

   Univ. of Amsterdam, Leiden Univ,Amsterdam,Netherlands

   Univ. of Amsterdam, Leiden Univ, Amsterdam, Netherlands

2. Quan Jiang

   Nanjing Agricultural University

   Affiliation as printed

   Nanjing Agricultural Univ,Nanjing,China

   Nanjing Agricultural Univ, Nanjing, China

3. Andy D. Pimentel

   University of Amsterdam

   Affiliation as printed

   Univ. of Amsterdam,Amsterdam,Netherlands

   Univ. of Amsterdam, Amsterdam, Netherlands

4. Todor P. Stefanov Aachen Leiden Univ. (Netherlands)

   Leiden University

   Affiliation as printed

   Leiden Univ,Leiden,Netherlands

   Leiden Univ, Leiden, Netherlands