ESCA Tutorial and Research Workshop on Speech Input/Output Assessment and Speech Databases Noordwijkerhout, The Netherlands September 20-23, 1989

Figures of Merit for Assessing Connected-Word Recognisers

Melvyn J. Hunt

Marconi Speech and Information Systems, The Airport, Portsmouth, UK

This paper is concerned mainly with the choice of a figure of merit for representing the performance of connected-word recognisers when DP word-symbol sequence matching is used for the scoring. Properties of the DP scoring method are discussed. Experimental tests using data from the DARPA Resource Management Task confirm a prediction that DP scoring overestimates substitution errors and underestimates insertion and deletion errors. As a result, the commonly used total error measure has a particularly large bias. A new figure of merit, weighted total errors, takes all three kinds of errors into account and minimises bias. Finally, some more sophisticated figures of merit are discussed briefly.

Full Paper

Bibliographic reference.  Hunt, Melvyn J. (1989): "Figures of merit for assessing connected-word recognisers", In SIOA-1989, Vol.2, 127-131.