Auditory-Visual Speech Processing (AVSP) 2013

Annecy, France
August 29 - September 1, 2013

Spontaneous Synchronisation between Repetitive Speech and Rhythmic Gesture

Gregory Zelic, Jeesun Kim, Chris Davis

The MARCS Institute, University of Western Sydney, Sydney, NSW, Australia

Although studies have described how motion in diverse biological systems may spontaneously synchronize it is not known whether speech and gesture exhibit such a property. Previous research on the coordination of speech and gesture has focused on pointing or tapping tasks, the structure of which may regulate speech and gesture dynamics. Here we examined whether synchronies might arise between a repetitive utterance and rhythmic finger movement oscillations in a non-intentional paradigm. Participants were instructed to repeatedly utter /ba/ or /sa/ syllables with/without vocalizing, while continuously moving their right index finger in flexion/extension. No instructions about synchronization were given; participants were only told to adopt the most comfortable motions. We expected that the larger amplitude of face motion for /ba/ syllables and vocalized speech would lead to greater influence on the gesture. In contrast, the results showed more synchronization for /sa/ and when syllables were articulated silently. Less perceptive feedback may lead to a reduction in the robustness of the speech component, making it more susceptible to gesture influence.

Index Terms: non-intentional synchronization, speech, gesture

Full Paper

Bibliographic reference.  Zelic, Gregory / Kim, Jeesun / Davis, Chris (2013): "Spontaneous synchronisation between repetitive speech and rhythmic gesture", In AVSP-2013, 17-20.