This paper is concerned mainly with the choice of a figure of merit for representing the performance of connected-word recognisers when DP word-symbol sequence matching is used for the scoring. Properties of the DP scoring method are discussed. Experimental tests using data from the DARPA Resource Management Task confirm a prediction that DP scoring overestimates substitution errors and underestimates insertion and deletion errors. As a result, the commonly used total error measure has a particularly large bias. A new figure of merit, weighted total errors, takes all three kinds of errors into account and minimises bias. Finally, some more sophisticated figures of merit are discussed briefly.
Cite as: Hunt, M.J. (1989) Figures of merit for assessing connected-word recognisers. Proc. Speech Input/Output Assessment and Speech Databases, Vol.2, 127-131
@inproceedings{hunt89_sioa,
author={Melvyn J. Hunt},
title={{Figures of merit for assessing connected-word recognisers}},
year=1989,
booktitle={Proc. Speech Input/Output Assessment and Speech Databases},
pages={Vol.2, 127-131}
}