8th Annual Conference of the International Speech Communication Association

Antwerp, Belgium
August 27-31, 2007

Acquisition and Synchronization of Multimodal Articulatory Data

Michael Aron, Nicolas Ferveur, Erwan Kerrien, Marie-Odile Berger, Yves Laprie

LORIA, France

This paper describes a setup to synchronize data used to track speech articulators during speech production. Our method couples together an ultrasound, an electromagnetic and an audio system to record speech sequences. The coupling requires a precise temporal synchronization, to know exactly the delay between the recording start of each modality, and to know the sampling rate of each modality. A complete setup and methods for automatically synchronizing data are described. The aim is to get a fast, low-cost and easily reproducible acquisition system in order to temporally align data.

Full Paper

Audio-Visual Example

aron_video_lowquality.avi  This video sequence shows the shape of the tongue seen on ultrasound images (white region, the apex is on the left), mixed with two electromagnetic sensors (green crosses) and with sound. The temporal synchronization between the three modalities can be visually checked.

Bibliographic reference.  Aron, Michael / Ferveur, Nicolas / Kerrien, Erwan / Berger, Marie-Odile / Laprie, Yves (2007): "Acquisition and synchronization of multimodal articulatory data", In INTERSPEECH-2007, 1398-1401.