Auditory-Visual Speech Processing (AVSP) 2009

University of East Anglia, Norwich, UK
September 10-13, 2009

Effects of Visual Prominence Cues on Speech Intelligibility

Samer Al Moubayed, Jonas Beskow

KTH Centre for Speech Technology, Stockholm, Sweden.

This study reports experimental results on the effect of visual prominence, presented as gestures, on speech intelligibility. 30 acoustically vocoded sentences, permutated into different gestural conditions were presented audio-visually to 12 subjects. The analysis of correct word recognition shows a significant increase in intelligibility when focally-accented (prominent) words are supplemented with head-nods or with eye-brow raise gestures. The paper also examines coupling other acoustic phenomena to brow-raise gestures. As a result, the paper introduces new evidence on the ability of the non-verbal movements in the visual modality to support audio-visual speech perception.

Index Terms: prominence, head-nod, eye-brow, speech intelligibility, talking heads, lip-reading, gesture, visual prosody.

Full Paper

Bibliographic reference.  Al Moubayed, Samer / Beskow, Jonas (2009): "Effects of visual prominence cues on speech intelligibility", In AVSP-2009, 43-46.