Auditory-Visual Speech Processing 2007 (AVSP2007) Kasteel Groenendaal, Hilvarenbeek, The Netherlands August 31 - September 3, 2007

Acoustic Effects of Visual Beats

Marc Swerts, Emiel Krahmer

Communication and Cognition, Tilburg University, The Netherlands

Speakers employ acoustic cues (pitch accents) to indicate that a word is important, but may also use visual cues (such as manual beat gestures, head nods, and eyebrow movements) for this purpose. Even though these acoustic and visual cues are related, the exact nature of this relationship is far from well understood. We investigate whether producing a visual beat leads to changes in how acoustic prominence is realized in speech. For this, we use an original experimental paradigm in which speakers are instructed to realize a target sentence with different distributions of acoustic and visual cues for prominence. Acoustic analyses reveal that the production of a visual beat indeed has an effect on the acoustic realization of the co-occuring speech, in particular on duration and the higher formants (F2 and F3), independent of the kind of visual beat and of the presence and position of pitch accents.

Full Paper

Bibliographic reference.  Swerts, Marc / Krahmer, Emiel (2007): "Acoustic effects of visual beats", In AVSP-2007, paper L4-2.