10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

Responding to User Emotional State by Adding Emotional Coloring to Utterances

Jaime C. Acosta, Nigel G. Ward

University of Texas at El Paso, USA

When people speak to each other, they share a rich set of nonverbal behaviors such as varying prosody in voice. These behaviors, sometimes interpreted as demonstrations of emotions, call for appropriate responses, but today’s spoken dialog systems lack the ability to do so. We collected a corpus of persuasive dialogs, specifically conversations about graduate school between a staff member and students, and had judges label all utterances with triples indicating the perceived emotions, using the three dimensions: activation, evaluation, and power. We found immediate response patterns, in which the staff member colored her utterances in response to the emotion shown by the student in the immediately previous utterance, and built a predictive model suitable for use in a dialog system to persuasively discuss graduate school with students.

Full Paper

Bibliographic reference.  Acosta, Jaime C. / Ward, Nigel G. (2009): "Responding to user emotional state by adding emotional coloring to utterances", In INTERSPEECH-2009, 1587-1590.