Auditory-Visual Speech Processing
A previous study by the first two authors suggests there is visual information for tone perception: under certain conditions Cantonese speakers are able to identify spoken words as one of six Cantonese words differing only in tone on the basis of lip and face movements at a rate better than chance . Here, non-native (tonal, Thai, and non-tonal, English) language speakers were tested on a discrimination version of this task in three modes: auditory-visual (AV), auditory only (AO), and visual only (VO). Auditory stimuli were presented either in the clear or accompanied by noise. In all conditions, even VO, performance was significantly better than chance. In the clear, both English and Thai perceivers performed better in the AO and AV conditions than in the VO condition. With auditory noise added, Thai perceivers performed better in the AV than the AO condition. The results support the existence of visual information for tone, and show that this is available in the absence of experience with the language in question, and even in the absence of experience with the lexical use of tone.
Bibliographic reference. Burnham, Denis / Lau, Susanna / Tam, Helen / Schoknecht, Colin (2001): "Visual discrimination of Cantonese tone by tonal but non-Cantonese speakers, and by non-tonal language speakers ", In AVSP-2001, 155-160.