Auditory-Visual Speech Processing (AVSP) 2011

Volterra, Italy
September 1-2, 2011

On the Audiovisual Asynchrony of Speech

László Czap

Department of Automation and Communication-Technology, University of Miskolc, Miskolc, Hungary

The temporal synchrony of auditory and visual signals is known to affect the perception of audiovisual speech. Several papers have discussed the asymmetry of acoustic and visual timing cues. These results are usually based on subjective intelligibility tests and the reason is remained obscure. It is not clear that the observation is perception or production origin. In this paper the effect of audio-visual asynchrony is studied in an automatic bimodal speech recognition task, eliminating the perception expertise of observers. Results are utilized to improve naturalness of audiovisual speech synthesis.

Index Terms. speechreading, multimodality, audiovisual asymmetry

Full Paper
Video 1 (mpg)    Video 2 (mpg)    Video 3 (avi)   

Bibliographic reference.  Czap, László (2011): "On the audiovisual asynchrony of speech", In AVSP-2011, 137-140.