ESCA - NATO/RSG 10 Workshop on
Applications of Speech Technology

Lautrach, Germany
September 16-17, 1993

Applying Multimodal Spoken Dialogue to Social-automation Systems

T. Nitta, Y. Masai, J. Iwasaki, S. Tanaka, H. Kamio, H. Matsu'uta

Info. Systems Eng. Lab., Toshiba Corp., Kawasaki, Japan

In this paper, we first briefly introduce voice input applications which we have developed during the last ten years. Then, after the discussion of some applications to social-automation systems, problems concerning the speech recognition capabilities and the user-system interaction facilities in practical environments are pointed out. Next, we describe a real-time multimodal dialogue system' which has been newly developed to overcome these problems. The system provides multiple input channels of spontaneous speech and touch, as well as multiple output channels of graphics and voice responses. The system also provides three sensors to detect the user's behavior and to plan interactive strategies. The experimental results using a map-guidance task are also discussed.

Full Paper

Bibliographic reference.  Nitta, T. / Masai, Y. / Iwasaki, J. / Tanaka, S. / Kamio, H. / Matsu'uta, H. (1993): "Applying multimodal spoken dialogue to social-automation systems", In AST-1993, 53-56.