Combining Feature and Model-Based Adaptation of RNNLMs for Multi-Genre Broadcast Speech Recognition

Salil Deena, Madina Hasan, Mortaza Doulaty, Oscar Saz, Thomas Hain


Recurrent neural network language models (RNNLMs) have consistently outperformed n-gram language models when used in automatic speech recognition (ASR). This is because RNNLMs provide robust parameter estimation through the use of a continuous-space representation of words, and can generally model longer context dependencies than n-grams. The adaptation of RNNLMs to new domains remains an active research area and the two main approaches are: feature-based adaptation, where the input to the RNNLM is augmented with auxiliary features; and model-based adaptation, which includes model fine-tuning and introduction of adaptation layer(s) in the network. This paper explores the properties of both types of adaptation on multi-genre broadcast speech recognition. Two hybrid adaptation techniques are proposed, namely the fine-tuning of feature-based RNNLMs and the use of a feature-based adaptation layer. A method for the semi-supervised adaptation of RNNLMs, using topic model-based genre classification, is also presented and investigated. The gains obtained with RNNLM adaptation on a system trained on 700h. of speech are consistent using both RNNLMs trained on a small (10Mwords) and large set (660M words), with 10% perplexity and 2% word error rate improvements on a 28.3h. test set.


DOI: 10.21437/Interspeech.2016-480

Cite as

Deena, S., Hasan, M., Doulaty, M., Saz, O., Hain, T. (2016) Combining Feature and Model-Based Adaptation of RNNLMs for Multi-Genre Broadcast Speech Recognition. Proc. Interspeech 2016, 2343-2347.

Bibtex
@inproceedings{Deena+2016,
author={Salil Deena and Madina Hasan and Mortaza Doulaty and Oscar Saz and Thomas Hain},
title={Combining Feature and Model-Based Adaptation of RNNLMs for Multi-Genre Broadcast Speech Recognition},
year=2016,
booktitle={Interspeech 2016},
doi={10.21437/Interspeech.2016-480},
url={http://dx.doi.org/10.21437/Interspeech.2016-480},
pages={2343--2347}
}