Sixth International Conference on Spoken Language Processing
Ratings of user satisfaction, although fairly easy to elicit for today's spoken language systems, can be more elusive for systems which operate at near-human levels of performance. This problem can be alleviated by adding a 're-listening' phase before eliciting judgenients: in this phase the user listens to a recording of himself interacting with the system while consulting a transcript of that interaction This technique allows more sensitive judgements of system quality by avoiding problems arising from attention limits.
Bibliographic reference. Tsukahara, Wataru / Ward, Nigel (2000): "Evaluating responsiveness in spoken dialog systems", In ICSLP-2000, vol.3, 1097-1100.