FPGA-based Deep Learning Inference Accelerators: Where Are We Standing?

ACM Transactions on Reconfigurable Technology and Systems, vol. 16, pp. 1–32

Abstract

Recently, artificial intelligence applications have become part of almost all emerging technologies around us. Neural networks, in particular, have shown significant advantages and have been widely adopted over other approaches in machine learning. In this context, high processing power is deemed a fundamental challenge and a persistent requirement. Recent solutions facing such a challenge deploy hardware platforms to provide high computing performance for neural networks and deep learning algorithms. This direction is also rapidly taking over the market. Here, FPGAs occupy the middle ground regarding flexibility, reconfigurability, and efficiency compared to general-purpose CPUs, GPUs, on one side, and manufactured ASICs on the other. FPGA-based accelerators exploit the features of FPGAs to increase the computing performance for specific algorithms and algorithm features. Filling a gap, we provide holistic benchmarking criteria and optimization techniques that work across several classes of deep learning implementations. This article summarizes the current state of deep learning hardware acceleration: More than 120 FPGA-based neural network accelerator designs are presented and evaluated based on a matrix of performance and acceleration criteria, and corresponding optimization techniques are presented and discussed. In addition, the evaluation criteria and optimization techniques are demonstrated by benchmarking ResNet-2 and LSTM-based accelerators.

Authors 6

1. Anouar Nechi

   University of Lübeck

   Affiliation as printed

   University of Lübeck, Germany

2. Lukas Groth

   University of Lübeck

   Affiliation as printed

   University of Lübeck, Germany

3. Saleh Mulhem

   University of Lübeck

   Affiliation as printed

   University of Lübeck, Germany

4. Farhad Merchant Aachen

   Newcastle University · RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Germany and Newcastle University, United Kingdom

5. Rainer Buchty

   University of Lübeck

   Affiliation as printed

   University of Lübeck, Germany

6. Mladen Bereković

   University of Lübeck

   Affiliation as printed

   University of Lübeck, Germany