10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

Modeling Mutual Influence of Interlocutor Emotion States in Dyadic Spoken Interactions

Chi-Chun Lee, Carlos Busso, Sungbok Lee, Shrikanth S. Narayanan

University of Southern California, USA

In dyadic human interactions, mutual influence ó a personís influence on the interacting partnerís behaviors ó is shown to be important and could be incorporated into the modeling framework in characterizing, and automatically recognizing the participantsí states. We propose a Dynamic Bayesian Network (DBN) to explicitly model the conditional dependency between two interacting partnersí emotion states in a dialog using data from the IEMOCAP corpus of expressive dyadic spoken interactions. Also, we focus on automatically computing the Valence-Activation emotion attributes to obtain a continuous characterization of the participantsí emotion flow. Our proposed DBN models the temporal dynamics of the emotion states as well as the mutual influence between speakers in a dialog. With speech based features, the proposed network improves classification accuracy by 3.67% absolute and 7.12% relative over the Gaussian Mixture Model (GMM) baseline on isolated turnby- turn emotion classification.

Full Paper

Bibliographic reference.  Lee, Chi-Chun / Busso, Carlos / Lee, Sungbok / Narayanan, Shrikanth S. (2009): "Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions", In INTERSPEECH-2009, 1983-1986.