ISCA Archive Interspeech 2009
ISCA Archive Interspeech 2009

A closer look at quality judgments of spoken dialog systems

Klaus-Peter Engelbrecht, Felix Hartard, Florian Gödde, Sebastian Möller

User judgments of Spoken Dialog Systems provide evaluators of such systems with a valid measure of their overall quality. Models for the automatic prediction of user judgments have been built, following the introduction of PARADISE [1]. Main applications are the comparison of systems, the analysis of parameters affecting quality, and the adoption of dialog management strategies. However, a common model which applies to different systems and users has not been found so far. With the aim of getting a closer insight into the quality-relevant characteristics of spoken interactions, an experiment was conducted where 25 users judged the same 5 dialogs. User judgments were collected after each dialog turn. The paper presents an analysis of the obtained results and some conclusions for future work.

doi: 10.21437/Interspeech.2009-712

Cite as: Engelbrecht, K.-P., Hartard, F., Gödde, F., Möller, S. (2009) A closer look at quality judgments of spoken dialog systems. Proc. Interspeech 2009, 2787-2790, doi: 10.21437/Interspeech.2009-712

  author={Klaus-Peter Engelbrecht and Felix Hartard and Florian Gödde and Sebastian Möller},
  title={{A closer look at quality judgments of spoken dialog systems}},
  booktitle={Proc. Interspeech 2009},