In this paper we tackle the problem of understanding references expressed through several different modes in a multimodal. dialogue system. We focus our attention on the two Natural Language and designation (by means of a mouse), for which we show that there exists two main strands of answers to this problem: the referential and the syntactical strand. We illustrate the first one from our experience within the Multiworks project and we show that both present some major drawbacks that they alone cannot solve. Moreover, since multimodal systems are usually based on NL, we show that the designation mode can also present some interesting properties which give it an important power of expression. This leads us to the conclusion that a multimodal dialogue system should be based on a unified architecture hinging on a multifold focus mechanism. Keywords: Multimodal dialogue, Natural Language, Focus, Reference.
Cite as: Gaiffe, B., Romary, L., Pierrel, J.-M. (1991) References in a multimodal dialogue: towards a unified processing. Proc. 2nd European Conference on Speech Communication and Technology (Eurospeech 1991), 1481-1485, doi: 10.21437/Eurospeech.1991-329
@inproceedings{gaiffe91_eurospeech,
author={B. Gaiffe and L. Romary and Jean-Marie Pierrel},
title={{References in a multimodal dialogue: towards a unified processing}},
year=1991,
booktitle={Proc. 2nd European Conference on Speech Communication and Technology (Eurospeech 1991)},
pages={1481--1485},
doi={10.21437/Eurospeech.1991-329}
}