5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Usability Evaluation of IVR Systems With DTMF and ASR

Cristina Delogu, Andrea Di Carlo, Paolo Rotundi, Danilo Sartori

Fondazione Ugo Bordoni, Italy

The paper presents an usability evaluation of 4 different prototypes of the same IVR application: three of them are automatic speech recognition (ASR) based and the other is Dual Tone Multi Frequency (DTMF) based. Our work consists of the automation of a service providing information about new facilities offered by Telecom Italia. The usability of the different prototypes has been evaluated through objective and subjective measures. Objective measures such as task completion and correctness, number of calls for task, transaction time, number of turns, and recognition accuracy have been obtained through the system's logfiles and the recorded speech utterances. To gather subjective measures the users were asked to fill in a questionnaire about their perception of the quality of the overall interaction, their effort in interacting with the system, and their satisfaction with different features of the system. In general a good correspondence between objective and subjective measures was found.

Full Paper

Bibliographic reference.  Delogu, Cristina / Carlo, Andrea Di / Rotundi, Paolo / Sartori, Danilo (1998): "Usability evaluation of IVR systems with DTMF and ASR", In ICSLP-1998, paper 0320.