Auditory-Visual Speech Processing (AVSP'98)

December 4-6, 1998
Terrigal - Sydney, Australia

Automatic Generation Of Cued Speech For The Deaf: Status and Outlook

Paul Duchnowski, Louis D. Braida, David Lum, Matthew Sexton, Jean Krause, Smriti Banthia

Massachusetts Institute of Technology, (USA)

Manual Cued Speech is a system of hand gestures designed to help deaf speechreaders distinguish among ambiguous speech elements. We have developed a computerized cueing system that uses automatic speech recognition to determine and display cues to the cue receiver. Keyword scores of 66% in low-context sentences have been obtained with this system, almost double the speechreading-alone scores. We describe the design issues with the largest impact on the cuer's performance, concentrating on the characteristics of cue display and cue timing. Enhancing cue images with color is found to improve their discriminability and may lead to improved speech reception by cue receivers.

Full Paper

Bibliographic reference.  Duchnowski, Paul / Braida, Louis D. / Lum, David / Sexton, Matthew / Krause, Jean / Banthia, Smriti (1998): "Automatic generation of cued speech for the deaf: status and outlook", In AVSP-1998, 161-166.

Multimedia Files

Link Original Filename Description Format
av98_161_1.mpg (140 KB) AVSP17_1.MPG Manually cued sentence, "The lawn grew around the flower bed." MPEG video file; 30 frames/sec, 320 x 240 size
av98_161_2.mpg (131 KB) AVSP17_2.MPG Automatically cued (discrete cues) sentence, "The low power made every light go dark." MPEG video file; 30 frames/sec, 320 x 240 size
av98_161_3.mpg (129 KB) AVSP17_3.MPG Automatically cued (dynamic+ cues) sentence, "Just use it up and order more later." MPEG video file; 30 frames/sec, 320 x 240 size
av98_161_4.mpg (152 KB) AVSP17_4.MPG Automatically cued (selectively colorized dynamic+ cues) sentence,"It takes little sugar to sweeten grapes." MPEG video file; 30 frames/sec, 320 x 240 size