Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition

arXiv (Cornell University)

Abstract

We study a streamable attention-based encoder-decoder model in which either the decoder, or both the encoder and decoder, operate on pre-defined, fixed-size windows called chunks. A special end-of-chunk (EOC) symbol advances from one chunk to the next chunk, effectively replacing the conventional end-of-sequence symbol. This modification, while minor, situates our model as equivalent to a transducer model that operates on chunks instead of frames, where EOC corresponds to the blank symbol. We further explore the remaining differences between a standard transducer and our model. Additionally, we examine relevant aspects such as long-form speech generalization, beam size, and length normalization. Through experiments on Librispeech and TED-LIUM-v2, and by concatenating consecutive sequences for long-form trials, we find that our streamable model maintains competitive performance compared to the non-streamable variant and generalizes very well to long-form speech.

Authors 4

1. Mohammad Zeineldeen Aachen Machine Learning and Human Language Technology Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   Machine Learning and Human Language Technology , Computer Science Department , RWTH Aachen University , Germany

2. Albert Zeyer Aachen Machine Learning and Human Language Technology Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   AppTek GmbH , Germany

   Machine Learning and Human Language Technology , Computer Science Department , RWTH Aachen University , Germany

3. Ralf Schlüter
4. Hermann Ney Aachen Machine Learning and Human Language Technology Computer Science Department

   RWTH Aachen University

   Affiliation as printed

   Machine Learning and Human Language Technology , Computer Science Department , RWTH Aachen University , Germany