End-to-End Speech Recognition: A Survey

IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 32, pp. 325–351

Abstract

In the last decade of automatic speech recognition (ASR) research, the introduction of deep learning has brought considerable reductions in word error rate of more than 50% relative, compared to modeling without deep learning. In the wake of this transition, a number of all-neural ASR architectures have been introduced. These so-calledend-to-end(E2E) models provide highly integrated, completely neural ASR models, which rely strongly on general machine learning knowledge, learn more consistently from data, with lower dependence on ASR domain-specific experience. The success and enthusiastic adoption of deep learning, accompanied by more generic model architectures has led to E2E models now becoming the prominent ASR approach. The goal of this survey is to provide a taxonomy of E2E ASR models and corresponding improvements, and to discuss their properties and their relationship to classical hidden Markov model (HMM) based ASR architectures. All relevant aspects of E2E ASR are covered in this work: modeling, training, decoding, and external language model integration, discussions of performance and deployment opportunities, as well as an outlook into potential future developments.

Authors 5

1. Rohit Prabhavalkar

   Google (United States)

   Affiliation as printed

   Google LLC., Mountain View, CA, USA

   Google Incorporated, Mountain View, CA, USA

2. Takaaki Hori

   Apple (United States)

   Affiliation as printed

   Apple Inc., Cambridge, MA, USA

3. Tara N. Sainath
   Affiliation as printed

   Google LLC., New York, NY, USA

4. Ralf Schlüter Aachen Lehrstuhl Informatik 6 - Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   Lehrstuhl Informatik 6 - Computer Science Department, RWTH Aachen University, Aachen, Germany

5. Shinji Watanabe

   Mitsubishi Electric Research Laboratories (United States) · Carnegie Mellon University

   Affiliation as printed

   Carnegie Mellon University, Pittsburgh, PA, USA

   Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA