Recurrent neural network language models (RNNLMs) can be augmented
with auxiliary features, which can provide an extra modality on top
of the words. It has been found that RNNLMs perform best when trained
on a large corpus of generic text and then fine-tuned on text corresponding
to the sub-domain for which it is to be applied. However, in many cases
the auxiliary features are available for the sub-domain text but not
for the generic text. In such cases, semi-supervised techniques can
be used to infer such features for the generic text data such that
the RNNLM can be trained and then fine-tuned on the available in-domain
data with corresponding auxiliary features.
In this paper, several
novel approaches are investigated for dealing with the semi-supervised
adaptation of RNNLMs with auxiliary features as input. These approaches
include: using zero features during training to mask the weights of
the feature sub-network; adding the feature sub-network only at the
time of fine-tuning; deriving the features using a parametric model
and; back-propagating to infer the features on the generic text. These
approaches are investigated and results are reported both in terms
of PPL and WER on a multi-genre broadcast ASR task.
Cite as: Deena, S., Ng, R.W.M., Madhyastha, P., Specia, L., Hain, T. (2017) Semi-Supervised Adaptation of RNNLMs by Fine-Tuning with Domain-Specific Auxiliary Features. Proc. Interspeech 2017, 2715-2719, doi: 10.21437/Interspeech.2017-1598
@inproceedings{deena17_interspeech,
author={Salil Deena and Raymond W.M. Ng and Pranava Madhyastha and Lucia Specia and Thomas Hain},
title={{Semi-Supervised Adaptation of RNNLMs by Fine-Tuning with Domain-Specific Auxiliary Features}},
year=2017,
booktitle={Proc. Interspeech 2017},
pages={2715--2719},
doi={10.21437/Interspeech.2017-1598}
}