International Conference on Auditory-Visual Speech Processing 2008

Tangalooma Wild Dolphin Resort, Moreton Island, Queensland, Australia
September 26-29, 2008

Perception of ‘Speech-and-Gesture’ Integration

Gianluca Giorgolo (1), Frans A. J. Verstraten (2)

(1) UiL-OTS, Universiteit Utrecht, The Netherlands
(2) Helmholtz Instituut, Experimental Psychology Division, Universiteit Utrecht, The Netherlands

This paper describes two experiments conducted to identify the role of synchronization in the perception of ‘speech and gesture’ communication and to isolate the parameters that determine the perception of temporal alignment. The results of the first experiment show that the synchronization between audio and visual signals determines the felicitousness of a multimodal utterance. With the second experiment we were able to determine that prosodic alignment is a parameter that our subjects used to judge the ‘well-formedness’ of speech and gesture input.

Full Paper

Bibliographic reference.  Giorgolo, Gianluca / Verstraten, Frans A. J. (2008): "Perception of ‘speech-and-gesture˛ integration", In AVSP-2008, 31-36.