Sixth European Conference on Speech Communication and Technology
We present the evaluation of the most recent version of the Dutch ARISE train timetable information system. The original version of this spoken dialogue system has been adjusted according to the findings of two user tests [1,2]. The new version applies a mixture of implicit and explicit confirmation of information items, based on confidence measures. In addition, the negotiation part of the dialogue tells the user explicitly what he can ask. Furthermore, the exceptions handling was made very explicit. The new dialogue has been evaluated by 25 experts and by 200+ anonymous calls to the system. To be able to compare the two versions of the system, the same scenarios as in  were used. It was shown that the mixture of implicit and explicit confirmation results in shorter dialogues and in slightly higher dialogue success rates. Also, we observed a better performance in the negotiation part of the dialogue. However, the shortcomings of working with complicated scenarios are once again made clear.
Full Paper (PDF) Gnu-Zipped Postscript
Bibliographic reference. Sturm, Janienke / Os, Els den / Boves, Lou (1999): "Dialogue management in the dutch ARISE train timetable information system", In EUROSPEECH'99, 1419-1422.