Sixth International Conference on Spoken Language Processing (ICSLP 2000)

Beijing, China
October 16-20, 2000

On the Correlation between Facial Movements, Tongue Movements and Speech Acoustics

Jintao Jiang (1), Abeer Alwan (1), Lynne E. Bernstein (2), Patricia Keating (3), Ed Auer (2)

(1) Electrical Engineering Department, University of California at Los Angeles, CA, USA
(2) Communication Neuroscience Department, House Ear Institute, Los Angeles, CA, USA
(3) Linguistics Department, University of California at Los Angeles, CA, USA 11232

This study is a first step in a large-scale study that aims at quantifying the relationship between external facial movements, tongue movements, and the acoustics of speech sounds. The database analyzed consisted of 69 CV syllables spoken by two males and two females; each utterance was repeated four times. A Qualysis (optical motion capture system) and an EMA (electromagnetic midsaggital articulography) system were used to characterize facial and tongue movements, respectively. Acoustic features were represented by linear spectral pairs (LSP). To quantify the correlation between them, a multilinear regression technique was applied. The results were analyzed in terms of vowel context, place of articulation, and individual articulatory (EMA or Optical) or acoustic (LSP) channel.


Full Paper

Bibliographic reference.  Jiang, Jintao / Alwan, Abeer / Bernstein, Lynne E. / Keating, Patricia / Auer, Ed (2000): "On the correlation between facial movements, tongue movements and speech acoustics", In ICSLP-2000, vol.1, 42-45.