12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

“You made me do it”: Classification of Blame in Married Couples' Interactions by Fusing Automatically Derived Speech and Language Information

Matthew P. Black, Panayiotis G. Georgiou, Athanasios Katsamanis, Brian R. Baucom, Shrikanth Narayanan

University of Southern California, USA

One of the goals of behavioral signal processing is the automatic prediction of relevant high-level human behaviors from complex, realistic interactions. In this work, we analyze dyadic discussions of married couples and try to classify extreme instances (low/high) of blame expressed from one spouse to another. Since blame can be conveyed through various communicative channels (e.g., speech, language, gestures), we compare two different classification methods in this paper. The first classifier is trained with the conventional static acoustic features and models "how" the spouses spoke. The second is a novel automatic speech recognition-derived classifier, which models "what" the spouses said. We get the best classification performance (82% accuracy) by exploiting the complementarity of these acoustic and lexical information sources through score-level fusion of the two classification methods.

Full Paper

Bibliographic reference.  Black, Matthew P. / Georgiou, Panayiotis G. / Katsamanis, Athanasios / Baucom, Brian R. / Narayanan, Shrikanth (2011): "“you made me do it”: classification of blame in married couples' interactions by fusing automatically derived speech and language information", In INTERSPEECH-2011, 89-92.