11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Recognition of Spontaneous Conversational Speech Using Long Short-Term Memory Phoneme Predictions

Martin Wöllmer, Florian Eyben, Björn Schuller, Gerhard Rigoll

Technische Universität München, Germany

We present a novel continuous speech recognition framework designed to unite the principles of triphone and Long Short-Term Memory (LSTM) modeling. The LSTM principle allows a recurrent neural network to store and to retrieve information over long time periods, which was shown to be well-suited for the modeling of co-articulation effects in human speech. Our system uses a bidirectional LSTM network to generate a phoneme prediction feature that is observed by a triphone-based large-vocabulary continuous speech recognition (LVCSR) decoder, together with conventional MFCC features. We evaluate both, phoneme prediction error rates of various network architectures and the word recognition performance of our Tandem approach using the COSINE database - a large corpus of conversational and noisy speech, and show that incorporating LSTM phoneme predictions in to an LVCSR system leads to significantly higher word accuracies.

Full Paper

Bibliographic reference.  Wöllmer, Martin / Eyben, Florian / Schuller, Björn / Rigoll, Gerhard (2010): "Recognition of spontaneous conversational speech using long short-term memory phoneme predictions", In INTERSPEECH-2010, 1946-1949.