ESCA Tutorial and Research Workshop on
This paper is concerned mainly with the choice of a figure of merit for representing the performance of connected-word recognisers when DP word-symbol sequence matching is used for the scoring. Properties of the DP scoring method are discussed. Experimental tests using data from the DARPA Resource Management Task confirm a prediction that DP scoring overestimates substitution errors and underestimates insertion and deletion errors. As a result, the commonly used total error measure has a particularly large bias. A new figure of merit, weighted total errors, takes all three kinds of errors into account and minimises bias. Finally, some more sophisticated figures of merit are discussed briefly.
Bibliographic reference. Hunt, Melvyn J. (1989): "Figures of merit for assessing connected-word recognisers", In SIOA-1989, Vol.2, 127-131.