It is well known that lip-reading is necessary for hearing impaired persons in order to (partially) understand speech, specifically, by using what is recoverable from visual speech. But as soon as in 1935, Cotton stated that there is an important element of visual hearing in all normal individuals. Even if the auditory modality is most important for speech perception by normal hearers, the visual modality may allow subjects to better understand speech. One must notice that visual information, provided by movements of the lips, the jaw, the teeth, etc., cannot, in itself, provide normal speech intelligibility. It has been demonstrated however that vision may almost entirely replace spectral information if the laryngeal timing is provided. Even tactile speech communication, such as the Tadoma method, where a hand is placed on the face of the talker, allows deaf-blind people to achieve almost normal communication. The vision of the talker's face more commonly enhances spectral information that is distorted by background noise. A number of investigators have studied this effect of distortion by noise on speech intelligibility according to whether the message is heard only, or heard with the speaker's face also provided.
Cite as: Benoît, C. (1991) A promising challenge for bimodal machine-man communication: the SYNTHESIS of TALKING FACES. Proc. 2nd VENACO Workshop - The Structure of Multimodal Dialogue, 169-172
@inproceedings{benoit91_smmd,
author={Christian Benoît},
title={{A promising challenge for bimodal machine-man communication: the SYNTHESIS of TALKING FACES}},
year=1991,
booktitle={Proc. 2nd VENACO Workshop - The Structure of Multimodal Dialogue},
pages={169--172}
}