ISCA Archive Interspeech 2021
ISCA Archive Interspeech 2021

Reducing Streaming ASR Model Delay with Self Alignment

Jaeyoung Kim, Han Lu, Anshuman Tripathi, Qian Zhang, Hasim Sak

Reducing prediction delay for streaming end-to-end ASR models with minimal performance regression is a challenging problem. Constrained alignment is a well-known existing approach that penalizes predicted word boundaries using external low-latency acoustic models. On the contrary, recently proposed FastEmit is a sequence-level delay regularization scheme encouraging vocabulary tokens over blanks without any reference alignments. Although all these schemes are successful in reducing delay, ASR word error rate (WER) often severely degrades after applying these delay constraining schemes. In this paper, we propose a novel delay constraining method, named self alignment. Self alignment does not require external alignment models. Instead, it utilizes Viterbi forced-alignments from the trained model to find the lower latency alignment direction. From LibriSpeech evaluation, self alignment outperformed existing schemes: 25% and 56% less delay compared to FastEmit and constrained alignment at the similar word error rate. For Voice Search evaluation, 12% and 25% delay reductions were achieved compared to FastEmit and constrained alignment with more than 2% WER improvements.


doi: 10.21437/Interspeech.2021-322

Cite as: Kim, J., Lu, H., Tripathi, A., Zhang, Q., Sak, H. (2021) Reducing Streaming ASR Model Delay with Self Alignment. Proc. Interspeech 2021, 3440-3444, doi: 10.21437/Interspeech.2021-322

@inproceedings{kim21j_interspeech,
  author={Jaeyoung Kim and Han Lu and Anshuman Tripathi and Qian Zhang and Hasim Sak},
  title={{Reducing Streaming ASR Model Delay with Self Alignment}},
  year=2021,
  booktitle={Proc. Interspeech 2021},
  pages={3440--3444},
  doi={10.21437/Interspeech.2021-322}
}