Auditory-Visual Speech Processing (AVSP) 2011

Volterra, Italy
September 1-2, 2011

Development of Communication Support System using Lip Reading

Takeshi Saitoh

Kyushu Institute of Technology, Fukuoka, Japan

This paper develops an entire communication support system for speech handicaps using lip reading. Our system is a high level system and consists of a face detector using Viola-jones method, lip detector based on active appearance model, utterance section detector, determined phrase recognizer using DP matching, camera controller, and message output. We implemented our system in the commercial laptop, and demonstrated it in the various situations of the sitting position, supine position, and holding position. We confirmed our system was able to be used in various situations.

Index Terms. communication support system, lip reading, fixed phrase recognition, real-time system.

Full Paper    Video (m4v; 12 MB)    Video (mkv; 14 MB)   

Bibliographic reference.  Saitoh, Takeshi (2011): "Development of communication support system using lip reading", In AVSP-2011, 117-122.