Third International Conference on Spoken Language Processing (ICSLP 94)

Yokohama, Japan
September 18-22, 1994

Listener Eye Movement Behavior During Audiovisual Speech Perception

Eric Vatikiotis-Bateson, Inge-Marie Eigsti, Sumio Yano

ATR Human Information Processing Research Laboratories, Kyoto, Japan

Results are presented for two experiments in which listener eye movements were recorded during video presentations of extended monologues. Monologues were presented with various levels of acoustic masking noise. The first experiment verified that the presence of visual stimuli enhanced intelligibility of the audiovisual stimuli. The second experiment showed a clear tendency for listeners of both Japanese and English to move their eyes less as perceptual difficulty increased, even when the image size of the stimuli was many times larger than life-size. Furthermore, preliminary results are discussed concerning the role of the eyes and mouth as targets for visual cues to speech communication, and an hypothesis for audiovisual enhancement of perception is discussed that integrates auditory localization and tracking of visual cues to speech.

Full Paper

Bibliographic reference.  Vatikiotis-Bateson, Eric / Eigsti, Inge-Marie / Yano, Sumio (1994): "Listener eye movement behavior during audiovisual speech perception", In ICSLP-1994, 527-530.