Auditory-Visual Speech Processing (AVSP'99)

August 7-10, 1999
Santa Cruz, CA, USA

Effect of Facial Brightness Reversal on Visual and Audiovisual Speech Perception

Rika Kanzaki (1), Ruth Campbell (2)

(1) ATR Human Information Processing Research Laboratories, Kyoto, Japan
(2) University College London, UK

To investigate the nature of facial information involved in visual and audiovisual speech perception, we examined the influence of brightness reversal on speechreading and the McGurk effect. Brightness reversed (photonegative) images are harder to 'read as faces' than images with normal brightness relations, but, when animated, maintain all the movement information of a speaking face. If speechreading were dependent solely on analysis of the dynamic features of a speaking face, this manipulation should have little effect on susceptibility to McGurk fusion illusions. Conversely, if speechreading made use primarily of visual end-state forms ('key frames'), then this manipulation should profoundly reduce susceptibility to McGurk illusions. Although the brightness reversed static images of non-labial articulations were hard to identify correctly, natural movement of brightness-reversed faces generated a McGurk effect, although it was weaker than for normal brightness faces. These results suggest that both dynamic (time-varying) and image quality (not time-varying) are involved in the visual perception of spoken language and that the strength of their relative influence depends on the demands of the task.

Full Paper

Bibliographic reference.  Kanzaki, Rika / Campbell, Ruth (1999): "Effect of facial brightness reversal on visual and audiovisual speech perception", In AVSP-1999, paper #8.