Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

A Framework for Evaluating Contextual Understanding

Hélène Bonneau-Maynard, L. Devillers

LIMSI-CNRS, Orsay, France

In this paper we propose and describe a framework for evaluating and diagnosing the understanding component of a spoken dialog system which can be applied to both literal and contextual understanding. We have observed in a previous experiment that contextual understanding performance is strongly correlated to the user satisfaction. The framework uses a glassbox approach to diagnose the interpretation modules. Results are given on a 1681 literal understanding test set and 100 contextual understanding test set.

Full Paper

Bibliographic reference.  Bonneau-Maynard, Hélène / Devillers, L. (2000): "A framework for evaluating contextual understanding", In ICSLP-2000, vol.3, 638-641.