8th European Conference on Speech Communication and Technology

Geneva, Switzerland
September 1-4, 2003


Assessment of Spoken Dialogue System Usability - What are We really Measuring?

Lars Bo Larsen

Aalborg University, Denmark

Speech based interfaces have not experienced the breakthrough many have predicted during the last decade. This paper attempts to clarify some of the reasons why by investigating the currently applied methods of usability evaluation. Usability attributes especially important for speech based interfaces are identified and discussed. It is shown that subjective measures (even for widespread evaluation schemes, such as PARADISE) are mostly done in an ad hoc manner and are rarely validated. A comparison is made between some well-known scales, and through an example application of the CCIR usability questionnaire it is shown how validation of the subjective measures can be performed.

Full Paper

Bibliographic reference.  Larsen, Lars Bo (2003): "Assessment of spoken dialogue system usability - what are we really measuring?", In EUROSPEECH-2003, 1945-1948.