Speech and Language Technology for Disabled Persons

Stockholm, Sweden
May 31 - June 2, 1993

Tactile Representation of Selected Acoustic-Phonetic Features for Use in Lipreading

Paul Dalsgaard (1), Ove Andersen (1), Viggo Moss Hansen (2)

(1) Speech Technology Centre, Aalborg University, Denmark
(2) Aalborg School for Deaf Children, Aalborg, Denmark

This presentation describes the configuration of and first results from a prototype system which supports profoundly deaf children in their lipreading. The prototype system combines the information given by lipreading with output from three tactile vibrators each positioned on a fingertip and with auditory information given via a bass-hearing aid, which amplifies the acoustic signal in the frequency range between 100 Hz and 4 kHz. Each of the tactile vibrators responds to the presence of a specific articulatory acoustic-phonetic feature being present in the acoustic speech signal. The acoustic-phonetic features are derived by Self-Organising Neural Networks, which have been stimulated and calibrated to respond to the selected vocalic features Front Round and Mid, and to the consonantal feature Alveolar. Preliminary results show that the individual features can be calculated with a fairly high accuracy as tested against a manually labelled reference test database given for one male speaker. Future work will include testing the system approach for more speakers and across sex.

Full Paper

Bibliographic reference.  Dalsgaard, Paul / Andersen, Ove / Hansen, Viggo Moss (1993): "Tactile representation of selected acoustic-phonetic features for use in lipreading", In SLTDP-93, 39-42.