9th Annual Conference of the International Speech Communication Association

Brisbane, Australia
September 22-26, 2008

The Effect of Auditory and Visual Degradation on Audiovisual Perception of Native and Non-Native Speakers

Valerie Hazan, Enid Li

University College London, UK

This study investigated the effect of visual and auditory degradation on the weighting given by listeners to visual cues for tokens produced by native and non-native speakers. 'Ba', 'da' and 'ga' syllables uttered by English and Chinese speakers were presented in auditory (A), visual (V) and congruent/incongruent audiovisual (AV) conditions to English listeners. In some conditions, the visual channel was degraded via Gaussian blurring, the auditory channel by pink noise, or both channels were degraded. In A or V conditions, degradation reduced intelligibility. In AV conditions with one degraded channel, the weighting of the other channel increased, with little effect of speaker-language. However, in clear or combined-degradation conditions, listeners gave greater weight to visual cues for non-native speakers. The weighting of auditory and visual cues is therefore highly flexible and there is strong support for a 'foreign-language effect' in visual weighting.

Full Paper

Bibliographic reference.  Hazan, Valerie / Li, Enid (2008): "The effect of auditory and visual degradation on audiovisual perception of native and non-native speakers", In INTERSPEECH-2008, 1191-1194.