In this study, the gating paradigm was extended to the visual modality of speech to investigate the influence of speech style on the information value of visual cues (lip movements) in spoken word identification. Two female native speakers (NSs) of American English (AE) participated in separate videotaped conversations (unscripted speech) with the author. Sentences were selected, transcribed, and later reproduced on videotape by the talkers as scripted speech. Through digital editing, target words were gated (using two-frame gates) and then presented to NSs for identification. Variables were modality of presentation: auditory-visual (AV) vs. auditory-only (A-only), speech style (unscripted vs. scripted), condition (sentence context vs. excised word), word length (1 or 2 syllables), and initial consonant visual category. Results revealed significantly earlier identification in AV presentation, with context, and in scripted speech but interactions varied between talkers. For both, there was a significant interaction between initial consonant, speech style and modality. Findings highlight the talker- and context-dependent nature of bimodal spoken language processing and contribute to an understanding of the influence of speech style and visual cues in word identification in connected speech.
Cite as: Hardison, D.M. (2001) Bimodal word identification: effects of modality, speech style, sentence and phonetic/visual context. Proc. Auditory-Visual Speech Processing, 161-166
@inproceedings{hardison01_avsp,
author={Debra M. Hardison},
title={{Bimodal word identification: effects of modality, speech style, sentence and phonetic/visual context}},
year=2001,
booktitle={Proc. Auditory-Visual Speech Processing},
pages={161--166}
}