Auditory-Visual Speech Processing
This paper describes an approach for the creation of natural audio-video speech signals from an acoustic speech signal or from a text input. It converts speech signals, which are taken from a microphone input, into a facial animation on the computer screen, displaying a static background image with movements of the mouth region and of the eyes. The video output is synchronized with the acoustic signal. LIPPS can also display a facial animation, which is driven by keyboard or text file input. Two applications are at hand, an intelligent audio-video telephone terminal and a training aid for lipreading. A prototype system has been implemented.
Bibliographic reference. Bothe, Hans-Heinrich (2001): "LIPPS - A visual telephone for hearing-impaired", In AVSP-2001, 199 (abstract).