International Conference on Auditory-Visual Speech Processing 2008

Tangalooma Wild Dolphin Resort, Moreton Island, Queensland, Australia
September 26-29, 2008

The Effects of Temporal Asynchrony on the Intelligibility of Accelerated Speech

Douglas S. Brungart (1), Nandini Iyer (2), Brian D. Simpson (1), Virginie van Wassenhove (3)

(1) Air Force Research Laboratory, Wright Patterson AFB, OH, USA
(2) General Dynamics, Dayton, OH, USA
(3) California Institute of Technology, Division of Biology, Pasadena, CA, USA

When the audio and visual portions of a speech stimulus are presented synchronously, the resulting enhancement in intelligibility is generally much larger than the one obtained when the audio and visual stimuli are presented sequentially. However, perfect synchronization is not required to obtain a substantial audiovisual (AV) benefit: many studies have shown that AV integration is maximum when the audio signal is slightly delayed relative to the visual signal, and other studies have shown that substantial AV intelligibility enhancement typically extends over a 250+ ms range of AV delay values, from a 50 ms lag in the visual stimulus to a 200 ms lag in the audio stimulus. In this study, artificially accelerated speech stimuli were used to examine the impact that speaking rate has on the characteristics of this temporal integration window. The results indicate that maximal AV enhancement occurs over a progressively narrower range of delay values when the speaking rate increases. The results for the fastest speaking rates also show that peak AV enhancement occurred at a larger AV delay value (150ms) than has been reported in previous studies. However, there was no conclusive evidence to suggest that the audio delay value for peak AV enhancement systematically changed with the speaking rate of the stimulus.

Full Paper

Bibliographic reference.  Brungart, Douglas S. / Iyer, Nandini / Simpson, Brian D. / Wassenhove, Virginie van (2008): "The effects of temporal asynchrony on the intelligibility of accelerated speech", In AVSP-2008, 19-24.