Auditory-Visual Speech Processing (AVSP) 2013

Annecy, France
August 29 - September 1, 2013

Assessing the Visual Speech Perception of Sampled-Based Talking Heads

Paula D. Paro Costa, José Mario De Martino

Department of Industrial Automation and Computer Engineering, School of Electrical and Computer Engineering, University of Campinas, Campinas, S˜ao Paulo, Brazil

Focusing on flexible applications for limited computing devices, this paper investigates the improvement on the visual speech perception obtained by the implicitly modeling of coarticulation on a sample-based talking head that is characterized by a compact image database and a morphing visemes synthesis strategy. Speech intelligibility tests were applied to assess the effectiveness of the proposed context-dependent visemes (CDV) model, comparing it to a simpler model that does not handle coarticulation. The results show that, when compared to the simpler model, the CDV approach improves speech intelligibility in situations in which the audio is degraded by noise. Moreover the CDVmodel achieves 80%to 90% of visual speech intelligibility of video of a real talker in the tested cases. Additionally, when the audio is heavily degraded by noise, the results suggest that the mechanisms that explain visual speech perception depends on the quality of the audible information.

Index Terms: facial animation, sample-based, 2D, speech intelligibility

Full Paper

Bibliographic reference.  Paro Costa, Paula D. / Martino, José Mario De (2013): "Assessing the visual speech perception of sampled-based talking heads", In AVSP-2013, 49-54.