Visual perception of speech through spectrogram reading has long been a subject of research, as an aid for the deaf or hearing impaired. Attributing the lack of success in this type of visual aid mainly to the static form of information presented by the spectrograms, this paper proposes a system of dynamic visualisation for speech sounds. This system samples a high resolved, auditory-based spectrogram, with a window of 20 milliseconds duration, so that exploiting the periodicity of the input sound, it produces a phase-locked sequence of images. This sequence is then animated at a rate of 50 images per second to produce a movie-like image displaying both the time-varying and time-independent information of the underlying sound. Results of several preliminary experiments for evaluation of the potential usefulness of the system for the deaf, undertaken by normal-hearing subjects, support the quick learning and persistence of the gestures for small sets of single words and motivate further investigations.
Cite as: Soltani-Farani, A.-A., Chilton, E.H.S., Shirley, R. (1998) Dynamical spectrogram, an aid for the deaf. Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998), paper 0438, doi: 10.21437/ICSLP.1998-788
@inproceedings{soltanifarani98_icslp,
author={Ali-Asghar Soltani-Farani and Edward H.S. Chilton and Robin Shirley},
title={{Dynamical spectrogram, an aid for the deaf}},
year=1998,
booktitle={Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998)},
pages={paper 0438},
doi={10.21437/ICSLP.1998-788}
}