Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Hearing by Eye: Visual Spatial Degradation and the McGurk Effect

John MacDonald, Soren Andersen, Talis Bachmann

Department of Psychology, University of Portsmouth, Portsmouth, Hampshire, UK

McGurk and MacDonald (Nature 264, 746-748, 1976) discovered that when a discrepancy is created between visual information from lip movements and speech information from the auditory channel then perceivers often report a percept that is neither the auditory or visual stimulus, an illusory response. This ‘McGurk’ effect is strong evidence that perceivers extract key information about a speech sound from concomitant visual articulation. This study investigates the effects of spatial quantisation on the McGurk effect. Participants (N=20) were presented with incongruous auditory-visual combinations of simple consonant vowel tokens. The visual stimulus was intact or had undergone various degrees of degradation through spatial quantisation. McGurk type responses were significantly influenced by levels of quantisation with more veridical auditory responses at the coarser levels of quantisation. However, even at the coarsest level of quantisation some McGurk type responses were reported.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  MacDonald, John / Andersen, Soren / Bachmann, Talis (1999): "Hearing by eye: visual spatial degradation and the mcgurk effect", In EUROSPEECH'99, 1283-1286.