From Quantitative Analysis to Synthesis of Efficient Binary Neural Networks

IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 93–100

Abstract

Binary Neural Networks (BNNs) offer an effective way to slash the cost of computation and memory accesses in inference. Recently, a plurality of ideas has been proposed, some of which are complementary while others are incompatible. This work presents a thorough review of state-of-the-art methods and an analysis of their computational cost based on the energy consumption of fixed-point, ternary and binary MAC vector operations. We derive an approach on how to systematically design a cost-efficient BNN. Our quantized LeNet and VGGNet architectures highlight the benefit of prudent capacity augmentation, with layer-wise ternarization providing best improvement of accuracy over μJ/dassificatíon in BNNs.

Authors 3

1. Tim Stadtmann Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen, Aachen, Germany

2. Cecilia Latotzke Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen, Aachen, Germany

3. Tobias Gemmeke Aachen Chair of Integrated Digital Systems and Circuit Design (IDS)

   RWTH Aachen University

   Affiliation as printed

   Chair of Integrated Digital Systems and Circuit Design, RWTH Aachen, Aachen, Germany