EUROSPEECH 2003 - INTERSPEECH 2003 8th European Conference on Speech Communication and Technology Geneva, Switzerland September 1-4, 2003

Three Simultaneous Speech Recognition by Integration of Active Audition and Face Recognition for Humanoid

Kazuhiro Nakadai (1), Daisuke Matsuura (2), Hiroshi G. Okuno (3), Hiroshi Tsujino (4)

(1) Japan Science and Technology Corporation, Japan
(2) Tokyo Institute of Technology, Japan
(3) Kyoto University, Japan
(4) Honda Research Institute Japan Co. Ltd., Japan

This paper addresses listening to three simultaneous talkers by a humanoid with two microphones. In such situations, sound separation and automatic speech recognition (ASR) of the separated speech are difficult, because the number of simultaneous talkers exceeds that of its microphones, the signal-to-noise ratio is quite low (around -3 dB) and noise is not stable due to interfering voices. Humanoid audition system consists of sound separation, face recognition and ASR. Sound sources are separated by an active direction-pass filter (ADPF), which extracts sounds from a specified direction in real-time. Since features of sounds separated by ADPF vary according to the sound direction, ASR uses multiple direction- and speaker-dependent acoustic models. The system integrates ASR results by using the sound direction and speaker information by face recognition as well as confidence measure of ASR results to select the best one. The resulting system improves word recognition rates against three simultaneous utterances.

Full Paper

Bibliographic reference.  Nakadai, Kazuhiro / Matsuura, Daisuke / Okuno, Hiroshi G. / Tsujino, Hiroshi (2003): "Three simultaneous speech recognition by integration of active audition and face recognition for humanoid", In EUROSPEECH-2003, 2705-2708.