INTERSPEECH 2011 12th Annual Conference of the International Speech Communication Association Florence, Italy August 27-31. 2011

Acoustic and Visual Cues of Turn-Taking Dynamics in Dyadic Interactions

Bo Xiao, Viktor Rozgić, Athanasios Katsamanis, Brian R. Baucom, Panayiotis G. Georgiou, Shrikanth Narayanan

University of Southern California, USA

In this paper we introduce an empirical study of multimodal cues of turn-taking dynamics in a social interaction context. We first identify pauses, gaps and overlapped speech segments in the dyadic conversation dataset. Second, we define two types of measurements, Mean Equalized Energy (MEE) and Animation Level (AL) on the audio and video channels, respectively. Then, we verify the hypothesis that the speaker with higher MEE or AL is more likely to take the floor after silence or overlapped speech. The results suggest that both the vocal and visual movement energy offer useful cues towards inferring the intention of the interlocutor to grab the floor.

Full Paper

Bibliographic reference.  Xiao, Bo / Rozgić, Viktor / Katsamanis, Athanasios / Baucom, Brian R. / Georgiou, Panayiotis G. / Narayanan, Shrikanth (2011): "Acoustic and visual cues of turn-taking dynamics in dyadic interactions", In INTERSPEECH-2011, 2441-2444.