International Symposium on Chinese Spoken Language Processing (ISCSLP 2002)

Taipei, Taiwan
August 23-24, 2002

Preparing for Evaluation of a Flight Spoken Dialogue System

Xiaojun Wu, Mingxing Xu, And Wenhu Wu

Center of Speech Technology, State Key Laboratory of Intelligent Technology and Systems, Tsinghua University, Beijing, China

Evaluation is needed to test how well the dialogue system works and is helpful for the developer to find some problems and to make the system more satisfactory. We have developed a flight spoken dialogue system and decide to carry out a thorough evaluation. Five dialogue scenarios with brief task descriptions are carefully designed. A questionnaire for user satisfaction is ready, which will be filled out by the subject immediately after the trial. The system will log some important information during the dialogue transactions, so that some objective metrics can be calculated. At last there are discussions about how to arrange the field trial and if there are any defects in the evaluation preparation.

Full Paper

Bibliographic reference.  Wu, Xiaojun / Xu, Mingxing / Wu, And Wenhu (2002): "Preparing for evaluation of a flight spoken dialogue system", In ISCSLP 2002, paper 50.