Auditory-Visual Speech Processing (AVSP'98)

December 4-6, 1998
Terrigal - Sydney, Australia

Lipreading Using Shape, Shading and Scale

Iain Matthews, Tim Cootes, Stephen Cox, Richard Harvey, J. Andrew Bangham

School of Information Systems, University of East Anglia, (U.K.)

This paper compares three methods of lipreading for visual and audio-visual speech recognition. Lip shape information is obtained using an Active Shape Model (ASM) lip tracker but is not as effective as modelling the combined shape and enclosed greylevel surface using an Active Appearance Model (AAM). A non-tracked alternative is a nonlinear transform of the image using a multiscale spatial analysis (MSA). This performs almost identically to AAM's in both visual and audio-visual recognition tasks on a multi-talker database of isolated letters.

Full Paper

Bibliographic reference.  Matthews, Iain / Cootes, Tim / Cox, Stephen / Harvey, Richard / Bangham, J. Andrew (1998): "Lipreading using shape, shading and scale", In AVSP-1998, 73-78.