Scalable Multi Corpora Neural Language Models for ASR

Anirudh Raju, Denis Filimonov, Gautam Tiwari, Guitang Lan, Ariya Rastrow


Neural language models (NLM) have been shown to outperform conventional n-gram language models by a substantial margin in Automatic Speech Recognition (ASR) and other tasks. There are, however, a number of challenges that need to be addressed for an NLM to be used in a practical large-scale ASR system. In this paper, we present solutions to some of the challenges, including training NLM from heterogenous corpora, limiting latency impact and handling personalized bias in the second-pass rescorer. Overall, we show that we can achieve a 6.2% relative WER reduction using neural LM in a second-pass n-best rescoring framework with a minimal increase in latency.


 DOI: 10.21437/Interspeech.2019-3060

Cite as: Raju, A., Filimonov, D., Tiwari, G., Lan, G., Rastrow, A. (2019) Scalable Multi Corpora Neural Language Models for ASR. Proc. Interspeech 2019, 3910-3914, DOI: 10.21437/Interspeech.2019-3060.


@inproceedings{Raju2019,
  author={Anirudh Raju and Denis Filimonov and Gautam Tiwari and Guitang Lan and Ariya Rastrow},
  title={{Scalable Multi Corpora Neural Language Models for ASR}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={3910--3914},
  doi={10.21437/Interspeech.2019-3060},
  url={http://dx.doi.org/10.21437/Interspeech.2019-3060}
}