ESCA Workshop on Audio-Visual Speech Processing (AVSP'97)

September 26-27, 1997
Rhodes, Greece

Preprocessing of Visual Speech under Real World Conditions

Uwe Meier, Rainer Stiefelhagen, Me Yang

Interactive Systems Laboratories, University of Karlsruhe, Karlsruhe, Germany Carnegie Mellon University, Pittsburgh, USA

In this paper we present recent work on integration of visual information (automatic lip-reading) with acoustic speech for better overall speech recognition. We have developed a modular system for flexible human-computer interaction via speech. In order to give the speaker reasonable freedom of movement within a room, the speaker's face is automatically acquired and followed by a face tracker subsystem, which delivers constant size, centered images of the face in real time. The image of the lips is automatically extracted from the camera image of the speaker's face by the lip tracker module, which can track the lips in real time. Furthermore, we show how the system deals with problems in real environments such as different illuminations and image sizes, and how the system adapts automatically to different noise conditions.

Full Paper

Bibliographic reference.  Meier, Uwe / Stiefelhagen, Rainer / Yang, Me (1997): "Preprocessing of visual speech under real world conditions", In AVSP-1997, 113-116.