Frame-Level MMI as A Sequence Discriminative Training Criterion for LVCSR

IEEE International Conference on Acoustics Speech and Signal Processing, vol. 15, pp. 6904–6908

Abstract

In this work we present frame-level maximum mutual information (MMI) as a novel sequence discriminative training criterion for hybrid HMM-DNN acoustic models. Compared to the standard, sequence-level MMI criterion we show that frame-level MMI has increased robustness towards missing cross-entropy (CE) smoothing and can converge even without interpolation. Using model free optimization, we show that in the asymptotic case of an infinite amount of training data models trained using this criterion are equal to the true class posterior distribution, whereas training using the state-level minimum Bayes risk (sMBR) criterion leads to a distorted function of the true class posterior distribution. This analytical result is backed by experimental evidence. We further propose a generalized class of training criteria, that continuously interpolates between frame-level MMI and sMBR criterion.

Authors 3

1. Wilfried Michel Aachen Human Language Technology and Pattern Recognition

   RWTH Aachen University

   Affiliation as printed

   Human Language Technology and Pattern Recognition, RWTH Aachen University, Aachen, Germany

2. Ralf Schlüter Aachen Human Language Technology and Pattern Recognition

   RWTH Aachen University

   Affiliation as printed

   Human Language Technology and Pattern Recognition, RWTH Aachen University, Aachen, Germany

3. Hermann Ney Aachen Human Language Technology and Pattern Recognition

   RWTH Aachen University

   Affiliation as printed

   Human Language Technology and Pattern Recognition, RWTH Aachen University, Aachen, Germany