Auditory-Visual Speech Processing (AVSP'98)

December 4-6, 1998
Terrigal - Sydney, Australia

Kinematics-Based Synthesis Of Realistic Talking Faces

Takaaki Kuratate, Hani Yehia, Eric Vatikiotis-Bateson

ATR Human Information Processing Research Laboratories, (Japan)

A method is described for animating talking faces that approach both cosmetic and communicative realism. The animations can be driven directly from a small set of time-varying positions measured on the face. This method of animation provides distinct benefits for both industrial and behavioral research applications, because the kinematic control parameters are easily obtained and are highly correlated with the measurable acoustic and neuromuscular events associated with speech production.


Full Paper

Bibliographic reference.  Kuratate, Takaaki / Yehia, Hani / Vatikiotis-Bateson, Eric (1998): "Kinematics-based synthesis of realistic talking faces", In AVSP-1998, 185-190.

Multimedia Files

Link Original Filename Description Format
av98_185_1.qt (1333 KB) eg98-short.qt Comparisons: original facial points, via-point interpolated facial points, original video, animated faces from original facial points and intepolated ones. Video File: QuickTime; Cinepak encoding, 320x240, 15frames/s, 11025Hz-16bit-mono audio
av98_185_2.qt (5043 KB) PCS.qt The facial deformation by each principal component Video File: QuickTime; Cinepak encoding, 320x240, 15frames/s, 11025Hz-16bit-mono audio
av98_185_3.qt (1398 KB) vnat005.qt Animation result with original voice and synthesized voice from face. Video File: QuickTime; Cinepak encoding, 320x240, 15frames/s, 11025Hz-16bit-mono audio
av98_185_4.qt (912 KB) cid023.qt Animation result (American subject) Video File: QuickTime; Cinepak encoding, 320x240, 15frames/s, 11025Hz-16bit-mono audio