ESCA Workshop on Audio-Visual Speech Processing (AVSP'97)
September 26-27, 1997
Humans perceive auditory [b] dubbed onto visual [g] as [d] or [D], as in 'them'. When this is presented with an [a] vowel, "th" responses tend to dominate, while in an [i] vowel context, "d" responses dominate. This "McGurk effect" was used here to investigate humans' integration of auditory and visual speech information. In Experiment 1, Australian English and Japanese subjects viewed McGurk stimuli presented by an English speaker. Despite the phonological irrelevance of [D] in Japanese, both English and Japanese subjects showed the [a]/[i] x "d" / "th" crossover effect, suggesting a strong language-general (phonetic) influence in auditory-visual integration. Experiment 2 used a Japanese speaker. Here the incidence of "th" responses for Japanese subjects was severely dampened, showing that expectancies based on native phonology may overlay basic phonetic auditory-visual integration.
Bibliographic reference. Burnham, Denis / Keane, Sheila (1997): "The Japanese Mcgurk effect: the role of linguistic and cultural factors an auditory-visual speech perception", In AVSP-1997, 93-96.