EUROSPEECH 2003 - INTERSPEECH 2003
Speech control is still mainly evaluated through statistical performance measures (recognition rate, insertion rate, etc.) considering the performance of a speech recognizer under laboratory or artificial noise conditions. All these measures give no idea about the practical usability of a speech interface, since practical aspects concern more than operational aspects of the speech recognizer inside a product. Since it was felt, that no evaluation standard so far fulfills the practical requirements for speech controlled products, this paper aims in the establishment of an open design- and evaluation standard for speech control concepts in real-world scenarios. First, the behaviour of the users and the normal environmental conditions (typical noises) were evaluated in usability experiments. Data recordings were conducted, trying to capture these typical usage requirements in a special corpus (Apollo-corpus). Finally, a set of standard desktop-, as well as embedded speech recognizers were tested for their performance under these real world conditions.
Bibliographic reference. Maase, Jens / Hirschfeld, Diane / Koloska, Uwe / Westfeld, Timo / Helbig, Jorg (2003): "Towards an evaluation standard for speech control concepts in real-world scenarios", In EUROSPEECH-2003, 1553-1556.