INTERSPEECH 2004 - ICSLP
8th International Conference on Spoken Language Processing

Jeju Island, Korea
October 4-8, 2004

Segmental Differences in the Visual Contribution to Speech Inteligibility

Kuniko Nielsen

UCLA, USA

It is well known that the presence of visual cues increases the overall intelligibility of a speech signal [1,2]. Although much is known about segmental differences in both audio-only and visual-only perception, little is known about segmental differences in terms of visual contribution to auditory-visual perception. The purpose of this study was to examine whether segments differ in their visual contribution to speech intelligibility, and whether the presence of visual cues always increases speech intelligibility. Forced-choice word-identification experiments were carried out under auditory-visual (AV) and auditory-only (A) conditions with varying S/N ratios. The experimental results reveal significant differences in the visual contribution for different consonants, with visual cues greatly improving speech intelligibility for most segments. Surprisingly, the results also suggest that the presence of visual cues can reduce intelligibility. In particular, the intelligibility of [r] decreased significantly in the AV condition, being perceived as [w] in most cases.

Full Paper

Bibliographic reference.  Nielsen, Kuniko (2004): "Segmental differences in the visual contribution to speech inteligibility", In INTERSPEECH-2004, 2533-2536.