Spoken language interfaces are rapidly improving in both speed and accuracy. However, miscommunication between system and user still occurs, with the potential for disrupting user/system interchange. This paper examines what happens when such miscommunication occurs: how quickly the user attempts to correct the system's non-understanding, what effect this has on subsequent communication, and how "expensive" miscommunication is in terms of solving a problem. We present a framework for analyzing detection and correction of system errors. Based on data from several experiments, we determined that in over 85% of the cases, the user detected a system misunderstanding in the next turn. For two systems with different overall performance (21% vs. 35% sentence understanding error) but identical response generation capabilities, errors took an average of 1.25 and 1.33 turns to detect, respectively; for a system with modified interaction and feedback strategies (and understanding error of 25%) the average dropped to 1.19 turns. This reduction in error detection time indicates that this measure is sensitive to aspects of the system/user interface beyond simple sentence understanding error.
Bibliographic reference. Hirschman, Lynette / Pao, Christine (1993): "The cost of errors in a spoken language system", In EUROSPEECH'93, 1419-1422.