Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Considerations in the Design and Evaluation of Spoken Language Dialog Systems

Lori Lamel, Sophie Rosset, Jean-Luc Gauvain

Spoken Language Processing Group, LIMSI-CNRS, Orsay, France

In this paper we summarize our experience at LIMSI in the design, development and evaluation of spoken language dialog systems for information retrieval tasks. This work has been for the most part carried out in the context of several European and international projects. Evaluation plays an integral role in the development of spoken language dialog systems. While there are commonly used measures and methodologies for evaluating speech recognizers, the evaluation of spoken dialog systems is considerably more complicated due to the interactive nature and the human perception of performance. It is therefore important to assess not only the individual system components, but the overall system performance using objective and subjective measures.

Full Paper

Bibliographic reference.  Lamel, Lori / Rosset, Sophie / Gauvain, Jean-Luc (2000): "Considerations in the design and evaluation of spoken language dialog systems", In ICSLP-2000, vol.4, 5-8.