International Symposium on Chinese Spoken Language Processing
August 23-24, 2002
Preparing for Evaluation of a Flight Spoken Dialogue System
Xiaojun Wu, Mingxing Xu, And Wenhu Wu
Center of Speech Technology, State Key Laboratory of Intelligent Technology and Systems,
Tsinghua University, Beijing, China
Evaluation is needed to test how well the dialogue system works
and is helpful for the developer to find some problems and to make
the system more satisfactory. We have developed a flight spoken
dialogue system and decide to carry out a thorough evaluation.
Five dialogue scenarios with brief task descriptions are carefully
designed. A questionnaire for user satisfaction is ready, which
will be filled out by the subject immediately after the trial.
The system will log some important information during the dialogue
transactions, so that some objective metrics can be calculated. At
last there are discussions about how to arrange the field trial and
if there are any defects in the evaluation preparation.
Wu, Xiaojun / Xu, Mingxing / Wu, And Wenhu (2002):
"Preparing for evaluation of a flight spoken dialogue system",
In ISCSLP 2002, paper 50.