12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

Detecting the Status of a Predictive Incremental Speech Understanding Model for Real-Time Decision-Making in a Spoken Dialogue System

David DeVault, Kenji Sagae, David Traum

University of Southern California, USA

We explore the potential for a responsive spoken dialogue system to use the real-time status of an incremental speech understanding model to guide its incremental decision-making about how to respond to a user utterance that is still in progress. Spoken dialogue systems have a range of potentially useful real-time response options as a user is speaking, such as providing acknowledgments or backchannels, interrupting the user to ask a clarification question or to initiate the system's response, or even completing the user's utterance at appropriate moments. However, implementing such incremental response capabilities seems to require that a system be able to assess its own level of understanding incrementally, so that an appropriate response can be selected at each moment. In this paper, we use a data-driven classification approach to explore the trade-offs that a virtual human dialogue system faces in reliably identifying how its understanding is progressing during a user utterance.

Full Paper

Bibliographic reference.  DeVault, David / Sagae, Kenji / Traum, David (2011): "Detecting the status of a predictive incremental speech understanding model for real-time decision-making in a spoken dialogue system", In INTERSPEECH-2011, 1021-1024.