Auditory-Visual Speech Processing (AVSP) 2010

Hakone, Kanagawa, Japan
September 30-October 3, 2010

Speech-Specificity of Two Audiovisual Integration Effects

Kasper Eskelund (1,3), Jyrki Tuomainen (2), Tobias S. Andersen (3),

(1) Center for Visual Cognition, Department of Psychology, University of Copenhagen, Denmark
(2) Speech Hearing and Phonetic Sciences, Division of Psychology and Language Sciences, University College London, UK
(3) Department of Informatics and Mathematical Modeling, Technical University of Denmark, Denmark

Seeing the talker’s articulatory mouth movements can influence the auditory speech percept both in speech identification and detection tasks. Here we show that these audiovisual integration effects also occur for sine wave speech (SWS), which is an impoverished speech signal that naïve observers often fail to perceive as speech. While audiovisual integration in the identification task only occurred when observers were informed of the speech-like nature of SWS, integration occurred in the detection task both for informed and naïve observers. This shows that both speech-specific and general mechanisms underlie audiovisual integration of speech.

Index Terms: Audiovisual integration, McGurk illusion, audiovisual speech detection

Full Paper

Bibliographic reference.  Eskelund, Kasper / Tuomainen, Jyrki / Andersen, Tobias S. / , Nina (2010): "Speech-specificity of two audiovisual integration effects", In AVSP-2010, paper P13.