5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Dynamical Spectrogram, an Aid for the Deaf

Ali-Asghar Soltani-Farani, Edward H.S. Chilton, Robin Shirley

University of Surrey, UK

Visual perception of speech through spectrogram reading has long been a subject of research, as an aid for the deaf or hearing impaired. Attributing the lack of success in this type of visual aid mainly to the static form of information presented by the spectrograms, this paper proposes a system of dynamic visualisation for speech sounds. This system samples a high resolved, auditory-based spectrogram, with a window of 20 milliseconds duration, so that exploiting the periodicity of the input sound, it produces a phase-locked sequence of images. This sequence is then animated at a rate of 50 images per second to produce a movie-like image displaying both the time-varying and time-independent information of the underlying sound. Results of several preliminary experiments for evaluation of the potential usefulness of the system for the deaf, undertaken by normal-hearing subjects, support the quick learning and persistence of the gestures for small sets of single words and motivate further investigations.

Full Paper

Bibliographic reference.  Soltani-Farani, Ali-Asghar / Chilton, Edward H.S. / Shirley, Robin (1998): "Dynamical spectrogram, an aid for the deaf", In ICSLP-1998, paper 0438.