Implementing neural networks in low-latency audio applications

The Journal of the Acoustical Society of America, vol. 153, pp. A105

Abstract

The use of neural networks is becoming increasingly prevalent due to their ability to represent complex relationships and solve complex problems. However, implementing these models in systems that require low-latency output can be challenging, especially for practitioners who are used to developing their models in controlled environments like Python notebooks. Another issue is the high computational cost of complex models, which limits the minimum possible latency. This paper presents approaches for deploying models in audio applications, discusses the advantages and disadvantages of each approach, and presents strategies to reduce the inference cost of models without significantly sacrificing accuracy, using techniques such as model quantization. To illustrate these methods, example implementations of real-time beamforming deconvolution and real-time music DSP processing are shown.

Authors 3

1. Thiago Henrique Gomes Lobato corresponding

   HEAD Acoustics (Germany)

   Affiliation as printed

   HEAD Acoust. GmbH, Ebertstraße 30a, Herzogenrath 52134, Germany, thiago.lobato@head-acoustics.de

2. Roland Sottek

   HEAD Acoustics (Germany)

   Affiliation as printed

   HEAD Acoust. GmbH, Herzogenrath, Germany

3. Michael Vorländer Aachen

   RWTH Aachen University

   Affiliation as printed

   IHTA, RWTH Aachen Univ., Aachen, Germany