9th Annual Conference of the International Speech Communication Association

Brisbane, Australia
September 22-26, 2008

Can Visualization of Internal Articulators Support Speech Perception?

Preben Wik, Olov Engwall

KTH, Sweden

This paper describes the contribution to speech perception given by animations of intra-oral articulations. 18 subjects were asked to identify the words in acoustically degraded sentences in three different presentation modes: acoustic signal only, audiovisual with a front view of a synthetic face and an audiovisual with both front face view and a side view, where tongue movements were visible by making parts of the cheek transparent. The augmented reality side-view did not help subjects perform better overall than with the front view only, but it seems to have been beneficial for the perception of palatal plosives, liquids and rhotics, especially in clusters. The results indicate that it cannot be expected that intra-oral animations support speech perception in general, but that information on some articulatory features can be extracted. Animations of tongue movements have hence more potential for use in computer-assisted pronunciation and perception training than as a communication aid for the hearing-impaired.

Full Paper

Bibliographic reference.  Wik, Preben / Engwall, Olov (2008): "Can visualization of internal articulators support speech perception?", In INTERSPEECH-2008, 2627-2630.