INTERSPEECH 2011 12th Annual Conference of the International Speech Communication Association Florence, Italy August 27-31. 2011

Auditory Speech Processing is Affected by Visual Speech in the Periphery

Jeesun Kim, Chris Davis

University of Western Sydney, Australia

Two experiments were conducted to determine whether visual speech presented in the visual periphery affects the perceived identity of speech sounds. Auditory speech targets (vCv syllables) were presented in noise (-8 dB) with congruent or incongruent visual speech presented in full-face or upper-half face conditions. Participants' eye-movements were monitored to assure that visual speech input occurred only from the periphery. In Experiment 1 participants had only to identify what they heard. The results showed that peripherally presented visual speech (full-face) facilitated identification of AV congruent stimuli compared to the upper-face control. Likewise, visual speech reduced correct identification for the incongruent stimuli. Experiment 2 was the same as the first except that in addition participants performed a central visual task. Once again significant effects of visual speech were found. These results show that peripheral visual speech affects speech recognition.

Full Paper

Bibliographic reference.  Kim, Jeesun / Davis, Chris (2011): "Auditory speech processing is affected by visual speech in the periphery", In INTERSPEECH-2011, 2465-2468.