FAAVSP - The 1st Joint Conference on Facial Analysis, Animation, and
Auditory-Visual Speech Processing

Vienna, Austria
September 11-13, 2015

Investigating the Impact of Artificial Enhancement of Lip Visibility on the Intelligibility of Spectrally-Distorted Speech

Najwa Alghamdi, Steve Maddock, Guy J. Brown, Jon Barker

Department of Computer Science, University of Sheffield, UK

The intelligibility of visual speech can be affected by a number of facial visual signals, e.g. lip emphasis, teeth and tongue visibility, and facial hair. This paper focuses on lip visibility. In the study presented in this paper, we use spectrally-distorted speech to train groups of non-native, English-speaking Saudi listeners using three different forms of speech: audio-only, audiovisual, and enhanced audiovisual, which is achieved by artificially colouring the lips of the speaker to improve lip visibility. The reason for using spectrally-distorted speech is that the longer term aim of our work is to employ these ideas in a training system for hearing-impaired users, in particular cochlear-implant users. Our initial work uses non-native Saudi listeners based on the assumption that their reduced processing abilities for native speech can be compared to the reduced processing abilities of cochlear implant users as a result of the inherent noise in the processing of sound by a cochlear implant. The results suggest that using enhanced audiovisual speech during auditory training improves the training gain when subsequently listening to audio-only spectrally-distorted speech. The results also suggest that spectrally-distorted speech intelligibility during training is improved when an enhanced visual signal is used.

Full Paper

Bibliographic reference.  Alghamdi, Najwa / Maddock, Steve / Brown, Guy J. / Barker, Jon (2015): "Investigating the impact of artificial enhancement of lip visibility on the intelligibility of spectrally-distorted speech", In FAAVSP-2015, 93-98.