Manual Cued Speech (MCS) is an effective method of communication by the deaf and hearing-impaired. We describe our work on assessing the feasibility of automatic determination and presentation of cues without intervention by the speaker. The conclusions of this study are applied to the design and implementation of a prototype automatic cueing system using HMM-based automatic speech recognition software to identify the cues in real time. We also describe the features of our cue display that enhance its effectiveness such as style of cue images and the timing of their transitions. Our experiments show keyword reception by experienced MCS users improves significantly with the use of our system (66%) relative to speechreading alone (35%) on low-context sentences.
Cite as: Duchnowski, P., Braida, L., Bratakos, M., Lum, D., Sexton, M., Krause, J. (1998) A speechreading aid based on phonetic ASR. Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998), paper 0589, doi: 10.21437/ICSLP.1998-820
@inproceedings{duchnowski98_icslp,
author={Paul Duchnowski and Louis Braida and Maroula Bratakos and David Lum and Matthew Sexton and Jean Krause},
title={{A speechreading aid based on phonetic ASR}},
year=1998,
booktitle={Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998)},
pages={paper 0589},
doi={10.21437/ICSLP.1998-820}
}