Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition

Interspeech (USB), pp. 2886–2890

Abstract

Subword units are commonly used for end-to-end automatic speech recognition (ASR), while a fully acoustic-oriented subword modeling approach is somewhat missing. We propose an acoustic data-driven subword modeling (ADSM) approach that adapts the advantages of several text-based and acoustic-based subword methods into one pipeline. With a fully acoustic-oriented label design and learning process, ADSM produces acoustic-structured subword units and acoustic-matched target sequence for further ASR training. The obtained ADSM labels are evaluated with different end-to-end ASR approaches including CTC, RNN-Transducer and attention models. Experiments on the LibriSpeech corpus show that ADSM clearly outperforms both byte pair encoding (BPE) and pronunciation-assisted subword modeling (PASM) in all cases. Detailed analysis shows that ADSM achieves acoustically more logical word segmentation and more balanced sequence length, and thus, is suitable for both time-synchronous and label-synchronous models. We also briefly describe how to apply acoustic-based subword regularization and unseen text segmentation using ADSM.

Authors 5

1. Wei Guo Zhou Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   AppTek GmbH, 52062 Aachen, Germany

   Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University, 52074 Aachen, Germany

2. Mohammad Zeineldeen Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   AppTek GmbH, 52062 Aachen, Germany

   Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University, 52074 Aachen, Germany

3. Zuoyun Zheng Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University, 52074 Aachen, Germany

4. Ralf Schlüter Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   AppTek GmbH, 52062 Aachen, Germany

   Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University, 52074 Aachen, Germany

5. Hermann Ney Aachen Human Language Technology and Pattern Recognition Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   AppTek GmbH, 52062 Aachen, Germany

   Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University, 52074 Aachen, Germany