Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Dialogue Management for Multimodal User Registration

Fei Huang, Jie Yang, Alex Waibel

Interactive System Lab, Carnegie Mellon University, Pittsburgh, PA, USA

User registration refers to associating certain personal information with a user. It is widely used in hospitals, hotels and conferences. In this paper, we propose an approach to interactive user registration by combining face recognition, speech recognition and speech synthesis technologies together through an efficient dialogue manager. In order to minimize a userís effort, we employ a new dialogue management model based on a finite state automaton (FSA), which uses a Baysian network to fuse the userís information from multiple channels (e.g., face image, speech, records stored in a pre-constructed database) to reliably estimate the confidence about user identity. Instead of fixing weights, the FSA adjusts its weights dynamically by integrating partial information from multiple information sources. This is achieved by maximizing an objective function to determine an optimal action at each succeeding state according to current confidence and information cues. Thus the transition between states can be done along the shortest path from the initial state to the goal state. We have developed a multimodal user registration system to demonstrate the feasibility of the proposed approach.


Full Paper

Bibliographic reference.  Huang, Fei / Yang, Jie / Waibel, Alex (2000): "Dialogue management for multimodal user registration", In ICSLP-2000, vol.3, 37-40.