Seeing the moving face of the talker permits better detection of speech in noise compared to not seeing their face. We report on an experiment that examined the basis of this AV facilitation effect. This work follows up research by [1] and [2] that developed a procedure for demonstrating an AV speech detection effect and [3] that showed that this facilitation occurred regardless of whether participants knew the language of test. In the current experiment we tested to see if AV facilitation occurred because participants were cued to when to pay attention by relatively simply properties of the visual speech of the talker (e.g., when the talkers mouth opened wide). This cuing idea was tested for two types of auditory and visual information that altered the naturalness of speech but maintained many simply cues. The first alteration was to present the AV stimuli backwards, e.g., (both speech and vision played timereversed). The second used a computer-generated face (Baldi) with synthesised speech. We also tested with a human talker with time-forward presentation. Our findings indicated that AV facilitation only occurred for the time forward human talker presentation; we discuss these results with respect to different types of Audio-Visual cuing.
s
Grant, K.W. / Seitz, P. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. Journal of the Acoustical Society of America, 108, 1197-1208. Grant, K.W. (2001). The effect of speechreading on masked detection thresholds for filtered speech. Journal of the Acoustical Society of America, 109, 2272-2275. Kim, J. & Davis, C. (2003). Hearing foreign voices: does knowing what is said affect masked visual speech detection? Perception, 32, 111-120.
Cite as: Kim, J., Davis, C. (2003) Testing the cuing hypothesis for the AV speech detection advantage. Proc. Auditory-Visual Speech Processing, 9-12
@inproceedings{kim03_avsp,
author={Jeesun Kim and Chris Davis},
title={{Testing the cuing hypothesis for the AV speech detection advantage}},
year=2003,
booktitle={Proc. Auditory-Visual Speech Processing},
pages={9--12}
}