Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Modeling the Prosody of Hidden Events for Improved Word Recognition

Andreas Stolcke, Elizabeth Shriberg, Dilek Hakkani-Tür, Gökhan Tür

Speech Technology and Research Laboratory, SRI International, Menlo Park, CA, USA

We investigate a new approach for using speech prosody as a knowledge source for speech recognition. The idea is to penal­ize word hypotheses that are inconsistent with prosodic features such as duration and pitch. To model the interaction between words and prosody we modify the language model to represent hidden events such as sentence boundaries and various forms of disfluency, and combine with it decision trees that predict such events from prosodic features. N­best rescoring experiments on the Switchboard corpus show a small but consistent reduction of word error as a result of this modeling. We conclude with a preliminary analysis of the types of errors that are corrected by the prosodically informed model.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Stolcke, Andreas / Shriberg, Elizabeth / Hakkani-Tür, Dilek / Tür, Gökhan (1999): "Modeling the prosody of hidden events for improved word recognition", In EUROSPEECH'99, 311-314.