ESCA Tutorial and Research Workshop on
Speech Input/Output Assessment and Speech Databases

Noordwijkerhout, The Netherlands
September 20-23, 1989

Testing the Effective Vocabulary Capacity Method of Evaluating Speech Recognizers

J. M. E. van de Vegte, M. M. Taylor

Defence and Civil Institute of Environmental Medicine, North York, Ontario, Canada

The performance of isolated-word speech recognizers is typically measured using error rates. To obtain reliable error rate estimates for good recognizers can require many thousands of subject utterances, and, the better the recognizer, the more sensitive the error rate is to the actual challenge vocabulary and the skill of the talker. The Effective Vocabulary Capacity (EVC) is the maximum vocabulary that a recognizer can in principle handle at a given error rate. It relies on measures that are relatively independent of the challenge vocabulary and which require only tens or hundreds of test utterances. The EVC algorithm is tested for both synthetic and real recognizer data.

Full Paper

Bibliographic reference.  Vegte, J. M. E. van de / Taylor, M. M. (1989): "Testing the effective vocabulary capacity method of evaluating speech recognizers", In SIOA-1989, Vol.2, 135-138.