ESCA Workshop on Audio-Visual Speech Processing (AVSP'97)

September 26-27, 1997
Rhodes, Greece

Video Rewrite: Visual Speech Synthesis from Video

Christoph Bregler, Michele Covell, Malcolm Slaney

Interval Research Corporation, Palo Alto, CA, USA

Video Rewrite uses existing footage to create automatically new video of a person mouthing words that she did not speak in the original footage. Video Rewrite uses computer-vision techniques to track points on the speaker's mouth in the training footage, and morphing techniques to combine these mouth gestures into the final video sequence. The new video combines the dynamics of the original actor's articulations with the mannerisms and setting dictated by the background footage. Video Rewrite is the first facial-animation system to automate all the labeling and assembly tasks required to resync existing footage to a new soundtrack.

Full Paper

Bibliographic reference.  Bregler, Christoph / Covell, Michele / Slaney, Malcolm (1997): "Video rewrite: visual speech synthesis from video", In AVSP-1997, 153-156.