Ratings of user satisfaction, although fairly easy to elicit for today's spoken language systems, can be more elusive for systems which operate at near-human levels of performance. This problem can be alleviated by adding a 're-listening' phase before eliciting judgenients: in this phase the user listens to a recording of himself interacting with the system while consulting a transcript of that interaction This technique allows more sensitive judgements of system quality by avoiding problems arising from attention limits.
Cite as: Tsukahara, W., Ward, N. (2000) Evaluating responsiveness in spoken dialog systems. Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), vol. 3, 1097-1100
@inproceedings{tsukahara00_icslp,
author={Wataru Tsukahara and Nigel Ward},
title={{Evaluating responsiveness in spoken dialog systems}},
year=2000,
booktitle={Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000)},
pages={vol. 3, 1097-1100}
}