7th International Conference on Spoken Language Processing

September 16-20, 2002
Denver, Colorado, USA

Gestural Trajectory Symmetries and Discourse Segmentation

Francis Quek (1), Yingen Xiong (1), David McNeill (2)

(1) Wright State University, USA; (2) University of Chicago, USA

Our approach is motivated by the conviction that gesture and speech are coexpressive of the underlying dynamic ideation that drives human communication. As such, transitions and cohesions is gestural behavior would inform us as to the discourse conceptualization. In this paper, we examine the role of motion symmetries of twohanded gestures in the structuring of speech. We employ a set of hand motion traces extracted from video and compute the correlation of these traces. The signs and magnitudes of the correlation coefficients computed in the cardinal directions of the subjectís torso (lateral and vertical in this work) characterize the symmetries. We employ a windowed computation approach that permits a balance between temporal resolution and robustness to noise. The resulting correlation profiles are merged according to a temporal proximity rule. We apply this analysis to two conversational video sequences. A detailed analysis of the first sequence reveals the persistence of gestural imagery between semantically-similar discourse pieces. A symmetry transition analysis is applied to the second dataset and compared against a manually generated discourse segmentation to demonstrate the potential of cross-modal discourse segmentation.

Full Paper

Bibliographic reference.  Quek, Francis / Xiong, Yingen / McNeill, David (2002): "Gestural trajectory symmetries and discourse segmentation", In ICSLP-2002, 185-188.