September 22-25, 1997
There are no agreed standards for reporting the performance of spoken dialogue systems. This paper proposes a core set of metrics to be used for this purpose. For this set, operational definitions are supplied, to regularise their application. The intention in proposing this framework is not that it should be exhaustive, nor that it should be perfect, but rather that it should provide a practical starting point, thereby allowing initial system comparison to be achieved quickly and with some measure of confidence.
Bibliographic reference. Fraser, Norman M. (1997): "Spoken dialogue system evaluation: a first framework for reporting results", In EUROSPEECH-1997, 1907-1910.