Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Robust Person Verification Based on Speech and Facial Images

J. Luettin, S. Ben-Yacoub

IDIAP, Martigny, Switzerland

This paper describes a multi-modal person verification system using speech and frontal face images. We consider two different speaker verification algorithms, a text-independent method using a second-order statistical measure and a text-dependent method based on hidden Markov modelling, as well as a face verification technique using a robust form of corellation. Fusion of the different recognition modules is performed by a Support Vector Machine classifier. Experimental results obtained on the audio-visual database XM2VTS for individual modalities and their combinations show that multimodal systems yield better performances than individual modules for all cases.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Luettin, J. / Ben-Yacoub, S. (1999): "Robust person verification based on speech and facial images", In EUROSPEECH'99, 991-994.