On Language Model Integration for RNN Transducer based Speech Recognition

arXiv (Cornell University)

Abstract

The mismatch between an external language model (LM) and the implicitly learned internal LM (ILM) of RNN-Transducer (RNN-T) can limit the performance of LM integration such as simple shallow fusion. A Bayesian interpretation suggests to remove this sequence prior as ILM correction. In this work, we study various ILM correction-based LM integration methods formulated in a common RNN-T framework. We provide a decoding interpretation on two major reasons for performance improvement with ILM correction, which is further experimentally verified with detailed analysis. We also propose an exact-ILM training framework by extending the proof given in the hybrid autoregressive transducer, which enables a theoretical justification for other ILM approaches. Systematic comparison is conducted for both in-domain and cross-domain evaluation on the Librispeech and TED-LIUM Release 2 corpora, respectively. Our proposed exact-ILM training can further improve the best ILM method.

Authors 4

1. Wei Guo Zhou Aachen AppTek GmbH Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University · FH Aachen

   Affiliation as printed

   AppTek GmbH , 52062 Aachen , Germany

   Human Language Technology and Pattern Recognition , Computer Science Department , RWTH Aachen University , 52074 Aachen , Germany

2. Zuoyun Zheng Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   Human Language Technology and Pattern Recognition , Computer Science Department , RWTH Aachen University , 52074 Aachen , Germany

3. Ralf Schlüter
4. Hermann Ney Aachen AppTek GmbH Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University · FH Aachen

   Affiliation as printed

   AppTek GmbH , 52062 Aachen , Germany

   Human Language Technology and Pattern Recognition , Computer Science Department , RWTH Aachen University , 52074 Aachen , Germany