Workshop on the Auditory Basis of Speech Perception Keele University, UK July 15-19, 1996

The Contribution of Temporally-Coded Acoustic Speech Patterns to Audio-Visual Speech Perception in Normally Hearing and Profoundly Hearing-Impaired Listeners

Andrew Faulkner, Stuart Rosen

Department of Phonetics and Linguistics, University College London, London, UK

Studies simulating hearing impairment in normally hearing subjects have investigated auditory and audio-visual consonant identification with acoustic signals representing isolated and combined temporal speech pattern elements. These elements comprise the temporal patterning of both periodic laryngeal excitation and aperiodic voiceless excitation, voice fundamental frequency, and the speech amplitude envelope. In consonant identification, the principal auditory contributions to audio-visual speech perception came from the on-and-off patterning of silence, periodic and aperiodic excitation. Variations in amplitude envelope and fundamental frequency provided little further information. In audio-visual sentence recognition, however, speech amplitude information did provide significant information beyond that from the temporal patterning and fundamental frequency of laryngeal excitation. A speech analysing hearing aid, the SiVo-II aid, has been employed to implement these codings. It employs an artificial neural-net classifier trained to extract laryngeal excitation information from speech in noise, and also extracts speech amplitude envelope. In a group of profoundly hearing- impaired listeners who derive little lipreading support from amplified speech, encoded speech pattern elements show a significant advantage, especially in noise.

Full Paper

Bibliographic reference.  Faulkner, Andrew / Rosen, Stuart (1996): "The contribution of temporally-coded acoustic speech patterns to audio-visual speech perception in normally hearing and profoundly hearing-impaired listeners", In ABSP-1996, 261-264.