ISCA Archive ICSLP 2000
ISCA Archive ICSLP 2000

Comparing the recognition performance of CSRs: in search of an adequate metric and statistical significance test

Helmer Strik, Catia Cucchiarini, Judith M. Kessens

In this paper a new measure of recognition accuracy is introduced which can be used when comparing the performance of two speech recognizers, to establish which is the better one. This metric combines the advantages of previous measures, but excludes their disadvantages. Essentially, the metric is an attempt to quantify the degree of recognition accuracy for each sentence, thus obtaining a more informative measure than either correct or incorrect, in such a way that the statistical significance of the observed differences can be tested. The advantages of our assessment method are illustrated on the basis of both artificial and real performance data of different recognizers.


Cite as: Strik, H., Cucchiarini, C., Kessens, J.M. (2000) Comparing the recognition performance of CSRs: in search of an adequate metric and statistical significance test. Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), vol. 4, 740-743

@inproceedings{strik00b_icslp,
  author={Helmer Strik and Catia Cucchiarini and Judith M. Kessens},
  title={{Comparing the recognition performance of CSRs: in search of an adequate metric and statistical significance test}},
  year=2000,
  booktitle={Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000)},
  pages={vol. 4, 740-743}
}