Sixth International Conference on Spoken Language Processing (ICSLP 2000)

Beijing, China
October 16-20, 2000

A Tool for the Synchronization of Speech and Mouth Shapes: LIPS

Odile Mella (1), Dominique Fohr (1), Laurent Martin (2), Andreas Carlen (2)

(1) LORIA-CNRS & INRIA Lorraine, Vandoeuvre-les-Nancy, France
(2)PROCOMA, Paris, France

This paper presents a new approach to improve the phoneme-based lipsync process. The lipsync process is a module in the animation production pipelines of 2D and 3D cartoons. It consists in generating the mouth positions of a cartoon character from dialogue recorded by an actor. The result is a sequence of time markers which indicate the series of mouth shapes to be drawn. We propose to speed up the lipsync process using tools coming from the field of automatic speech recognition. We describe the LIPS tool (LIPS Logiciel Interactif de PostSynchronisation: lipsync Interactive Software) and the generation of the acoustic models required by the tool and we present results obtained on two cartoons. Using the LIPS tool for the post-synchronization of 52 minutes of cartoon reduced the step of post-synchronization from 14 days to 3 days.

Full Paper

Bibliographic reference.  Mella, Odile / Fohr, Dominique / Martin, Laurent / Carlen, Andreas (2000): "A tool for the synchronization of speech and mouth shapes: LIPS", In ICSLP-2000, vol.1, 90-93.