AVSP 2003 - International Conference on Audio-Visual Speech Processing September 4-7, 2003 St. Jorioz, France

Discrimination of Auditory-Visual Synchrony

Ken W. Grant (1), Virginie van Wassenhove (2), David Poeppel (2)

(1) Army Audiology and Speech Center, Walter Reed Army Medical Center, Washington, DC, USA
(2) Neuroscience and Cognitive Science Program, Cognitive Neuroscience of Language Laboratory, University of Maryland, College Park, MD, USA

Discrimination thresholds for temporal synchrony in auditory-visual sentence materials were obtained on a group of normal-hearing subjects. Thresholds were determined using an adaptive tracking procedure which controlled the degree of audio delay, both positive and negative in separate tracks, relative to a video image of a female speaker. Four different auditory filter conditions, as well as a broadband speech condition, were evaluated in order to determine whether discrimination thresholds were dependent on the spectral content of the acoustic speech signal. Consistent with previous studies of auditory-visual speech recognition which showed a broad, asymmetrical range of temporal synchrony (audio delays roughly between -40 ms and +240 ms) for which intelligibility was basically unaffected, synchrony discrimination thresholds also showed a broad, asymmetrical pattern of similar magnitude (audio delays roughly between -45 ms and 200 ms). No differences in synchrony thresholds were observed for the different filtered bands of speech, or for broadband speech. These results suggest a fairly tight coupling between a subject's ability to detect cross-modal asynchrony and the intelligibility of auditory-visual speech materials.

Full Paper

Bibliographic reference.  Grant, Ken W. / Wassenhove, Virginie van / Poeppel, David (2003): "Discrimination of auditory-visual synchrony", In AVSP 2003, 31-35.