Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Intensity- and Location-Normalized Training for HMM-Based Visual Speech Recognition

Yoshihiko Nankaku, Keiichi Tokuda, Tadashi Kitamura

Department of Computer Science, Nagoya Institute of Technology, Japan

This paper describes an approach to estimating the pa-rameters of continuous density HMMs for visual speech recognition. One of the key issues of image-based vi-sual speech recognition is normalization of lip location and lighting condition prior to estimating the parameters of HMMs. We present an average-intensity and loca-tion normalized training method, in which the normalization process is integrated in the model training. The proposed method provides a theoretically-well-defined algorithm based on a maximum likelihood formulation, hence the likelihood for the training data is guaranteed to increase at each iteration of the normalized training. Experimental results show that the recognition performance can be significantly improved by the normalized training.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Nankaku, Yoshihiko / Tokuda, Keiichi / Kitamura, Tadashi (1999): "Intensity- and location-normalized training for HMM-based visual speech recognition", In EUROSPEECH'99, 1287-1290.