15th Annual Conference of the International Speech Communication Association

September 14-18, 2014

3D Tongue Motion Visualization Based on Ultrasound Image Sequences

Kele Xu (1), Yin Yang (2), A. Jaumard-Hakoun (1), Martine Adda-Decker (3), A. Amelot (3), S. K. Al Kork (1), L. Crevier-Buchman (3), P. Chawah (3), G. Dreyfus (4), T. Fux (3), C. Pillot-Loiseau (3), P. Roussel (4), M. Stone (5), B. Denby (1)

(1) UPMC, France
(2) University of New Mexico, USA
(3) LPP (UMR 7018), France
(4) SIGMA (UMR 7084), France
(5) University of Maryland, USA

The article proposes a real-time technique for visualizing tongue motion driven by ultrasound image sequences. Local feature description is used to follow characteristic speckle patterns in a set of mid-sagittal contour points in an ultrasound image sequence, which are then used as markers for describing movements of the tongue. A 3D tongue model is subsequently driven by the motion data extracted from the ultrasound image sequences. The “modal warping” technique is used for real-time tongue deformation visualization. The resulting system will be useful in a variety of domains including speech production study, articulation training, educational scenarios, etc. Some parts of the interface are still being developed; we will show preliminary results in the demonstration.

Full Paper

Bibliographic reference.  Xu, Kele / Yang, Yin / Jaumard-Hakoun, A. / Adda-Decker, Martine / Amelot, A. / Kork, S. K. Al / Crevier-Buchman, L. / Chawah, P. / Dreyfus, G. / Fux, T. / Pillot-Loiseau, C. / Roussel, P. / Stone, M. / Denby, B. (2014): "3d tongue motion visualization based on ultrasound image sequences", In INTERSPEECH-2014, 1482-1483.