An efficient method to reduce the amount of feature data for real-time automatic image-transform-based lipreading is proposed. Image-transform-based approach obtaining a compressed representation of image pixel values of speaker's mouth is reported to show superior lipreading performance. However, since this approach produces many feature vectors relevant to lip information, it requires much computation time for lipreading even when principal component analysis (PCA) is applied. To reduce the computational load efficiently, we propose an algorithm that utilizes the symmetry of the lip. The proposed method reduces the amount of required feature vectors up to 51% compared to the original one. Also, it improves the recognition rates by compensating the variation of illumination. With our database (22 words, 70 talkers) recorded in a natural environment, our method achieved an accuracy of 53.5% for visual-only speaker independent word recognition task. The extracted features are modeled by hidden Markov models with Gaussian mixture distributions.
Cite as: Lee, J., Kim, J. (2001) An efficient lipreading method using the symmetry of lip. Proc. 7th European Conference on Speech Communication and Technology (Eurospeech 2001), 1019-1022, doi: 10.21437/Eurospeech.2001-292
@inproceedings{lee01b_eurospeech,
author={Joohun Lee and JinYoung Kim},
title={{An efficient lipreading method using the symmetry of lip}},
year=2001,
booktitle={Proc. 7th European Conference on Speech Communication and Technology (Eurospeech 2001)},
pages={1019--1022},
doi={10.21437/Eurospeech.2001-292}
}