ESCA Workshop on Audio-Visual Speech Processing (AVSP'97) September 26-27, 1997 Rhodes, Greece

The Japanese McGurk Effect: The Role of Linguistic and Cultural Factors an Auditory-Visual Speech Perception

Denis Burnham, Sheila Keane

School of Psychology, University of NSW, Sydney, Australia

Humans perceive auditory [b] dubbed onto visual [g] as [d] or [D], as in 'them'. When this is presented with an [a] vowel, "th" responses tend to dominate, while in an [i] vowel context, "d" responses dominate. This "McGurk effect" was used here to investigate humans' integration of auditory and visual speech information. In Experiment 1, Australian English and Japanese subjects viewed McGurk stimuli presented by an English speaker. Despite the phonological irrelevance of [D] in Japanese, both English and Japanese subjects showed the [a]/[i] x "d" / "th" crossover effect, suggesting a strong language-general (phonetic) influence in auditory-visual integration. Experiment 2 used a Japanese speaker. Here the incidence of "th" responses for Japanese subjects was severely dampened, showing that expectancies based on native phonology may overlay basic phonetic auditory-visual integration.

Full Paper

Bibliographic reference.  Burnham, Denis / Keane, Sheila (1997): "The Japanese Mcgurk effect: the role of linguistic and cultural factors an auditory-visual speech perception", In AVSP-1997, 93-96.