An Online Attention-Based Model for Speech Recognition

Ruchao Fan, Pan Zhou, Wei Chen, Jia Jia, Gang Liu


Attention-based end-to-end models such as Listen, Attend and Spell (LAS), simplify the whole pipeline of traditional automatic speech recognition (ASR) systems and become popular in the field of speech recognition. In previous work, researchers have shown that such architectures can acquire comparable results to state-of-the-art ASR systems, especially when using a bidirectional encoder and global soft attention (GSA) mechanism. However, bidirectional encoder and GSA are two obstacles for real-time speech recognition. In this work, we aim to stream LAS baseline by removing the above two obstacles. On the encoder side, we use a latency-controlled (LC) bidirectional structure to reduce the delay of forward computation. Meanwhile, an adaptive monotonic chunk-wise attention (AMoChA) mechanism is proposed to replace GSA for the calculation of attention weight distribution. Furthermore, we propose two methods to alleviate the huge performance degradation when combining LC and AMoChA. Finally, we successfully acquire an online LAS model, LC-AMoChA, which has only 3.5% relative performance reduction to LAS baseline on our internal Mandarin corpus.


 DOI: 10.21437/Interspeech.2019-2218

Cite as: Fan, R., Zhou, P., Chen, W., Jia, J., Liu, G. (2019) An Online Attention-Based Model for Speech Recognition. Proc. Interspeech 2019, 4390-4394, DOI: 10.21437/Interspeech.2019-2218.


@inproceedings{Fan2019,
  author={Ruchao Fan and Pan Zhou and Wei Chen and Jia Jia and Gang Liu},
  title={{An Online Attention-Based Model for Speech Recognition}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={4390--4394},
  doi={10.21437/Interspeech.2019-2218},
  url={http://dx.doi.org/10.21437/Interspeech.2019-2218}
}