The purpose of this study was to investigate whether integration of audiovisual speech occurs automatically so that information from heard speech and seen articulatory movements of the talking face are combined without any voluntary effort. The McGurk effect, where seeing discrepant visual speech changes the auditory speech percept, was used as a tool since it reflects the extent of audiovisual integration. The McGurk effect was measured in two conditions which manipulated the subjects' attention to visual speech. In both conditions, the subjects' task was to attend to auditory speech and report what the talker said. In the 'Attend Face' condition, subjects were instructed to pay attention also to the talking face, presented in synchrony with auditory speech. In the 'Ignore Face' condition, subjects were instructed to ignore the talking face and to pay attention to a visual distractor presented in the same location as the face. The proportion of auditory responses was higher in the latter condition, indicating that the influence of visual speech was weaker when the face was not attended. This result suggests that integration of audiovisual speech is not entirely automatic. The mechanism underlying this attentional effect was investigated by fitting the Fuzzy Logical Model of Perception (FLMP) [1] to the results, and the good fit of the model implies that attention influences unimodal information processing before integration across modalities takes place.
Cite as: Tiippana, K., Sams, M., Andersen, T.S. (2001) Visual attention influences audiovisual speech perception. Proc. Auditory-Visual Speech Processing, 167-171
@inproceedings{tiippana01_avsp,
author={K. Tiippana and M. Sams and T. S. Andersen},
title={{Visual attention influences audiovisual speech perception}},
year=2001,
booktitle={Proc. Auditory-Visual Speech Processing},
pages={167--171}
}