ISCA Archive Interspeech 2017
ISCA Archive Interspeech 2017

An Exploration of Dropout with LSTMs

Gaofeng Cheng, Vijayaditya Peddinti, Daniel Povey, Vimal Manohar, Sanjeev Khudanpur, Yonghong Yan

Long Short-Term Memory networks (LSTMs) are a component of many state-of-the-art DNN-based speech recognition systems. Dropout is a popular method to improve generalization in DNN training. In this paper we describe extensive experiments in which we investigated the best way to combine dropout with LSTMs — specifically, projected LSTMs (LSTMP). We investigated various locations in the LSTM to place the dropout (and various combinations of locations), and a variety of dropout schedules. Our optimized recipe gives consistent improvements in WER across a range of datasets, including Switchboard, TED-LIUM and AMI.

doi: 10.21437/Interspeech.2017-129

Cite as: Cheng, G., Peddinti, V., Povey, D., Manohar, V., Khudanpur, S., Yan, Y. (2017) An Exploration of Dropout with LSTMs. Proc. Interspeech 2017, 1586-1590, doi: 10.21437/Interspeech.2017-129

  author={Gaofeng Cheng and Vijayaditya Peddinti and Daniel Povey and Vimal Manohar and Sanjeev Khudanpur and Yonghong Yan},
  title={{An Exploration of Dropout with LSTMs}},
  booktitle={Proc. Interspeech 2017},