11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Acoustic-Based Recognition of Head Gestures Accompanying Speech

Akira Sasou, Yasuharu Hashimoto, Katsuhiko Sakaue

AIST, Japan

Head movements are linked not only to symbolic gestures, such as head-nodding to represent “yes” or head-shaking to represent “no,” but also to the production of suprasegmental features of speech, such as stress, prominence, and other aspects of prosody. Recent studies have shown that head movements play a more direct role in the perception of speech. In this paper, we propose a novel method for recognizing head gestures that accompany speech. The proposed method tracks head movements that accompany speech by localizing the mouth position with a microphone array system. We also propose a recognition method for the mouth-position trajectory, in which Higher- Order Local Cross Correlation is applied to the trajectory. The recognition accuracy of the proposed method was on an average 90.25% for nineteen kinds of head gesture recognition tasks conducted in an open test manner, which outperformed the Hidden Markov Model-based method.

Full Paper

Bibliographic reference.  Sasou, Akira / Hashimoto, Yasuharu / Sakaue, Katsuhiko (2010): "Acoustic-based recognition of head gestures accompanying speech", In INTERSPEECH-2010, 506-509.