Auditory-Visual Speech Processing 2007 (AVSP2007)

Kasteel Groenendaal, Hilvarenbeek, The Netherlands
August 31 - September 3, 2007

Towards Eye Gaze Aware Analysis and Synthesis of Audiovisual Speech

Frédéric Elisei, Gérard Bailly, Alix Casari, Stephan Raidt

Departement "parole et cognition", GIPSA-Lab, Grenoble Universités, France

Eye gaze plays many important roles in audiovisual speech, especially in face-to-face interactions. Eyelid shapes are known to correlate with gaze direction. This correlation is perceived and should be restored when animating 3D talking heads. This paper presents a data-based construction method that models the user’s eyelid geometric deformations caused by gazing and blinking during conversation. This 3D eyelid and gaze model has been used to analyze and automatically reconstruct our German speaker’s gaze. This can potentially complement or replace infra-red based eye tracking when it is important to collect not only where the user looks but also how (ocular expressions). This method may be used as a tool to study expressive speech and gaze patterns related to cognitive activities (speaking, listening, thinking...).

Full Paper

Bibliographic reference.  Elisei, Frédéric / Bailly, Gérard / Casari, Alix / Raidt, Stephan (2007): "Towards eye gaze aware analysis and synthesis of audiovisual speech", In AVSP-2007, paper L2-3.