Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Comparing the Recognition Performance of CSRs: In Search of an Adequate Metric and Statistical Significance Test

Helmer Strik, Catia Cucchiarini, Judith M. Kessens

A2RT, Dept. of Language & Speech, University of Nijmegen, The Netherlands

In this paper a new measure of recognition accuracy is introduced which can be used when comparing the performance of two speech recognizers, to establish which is the better one. This metric combines the advantages of previous measures, but excludes their disadvantages. Essentially, the metric is an attempt to quantify the degree of recognition accuracy for each sentence, thus obtaining a more informative measure than either correct or incorrect, in such a way that the statistical significance of the observed differences can be tested. The advantages of our assessment method are illustrated on the basis of both artificial and real performance data of different recognizers.

Full Paper

Bibliographic reference.  Strik, Helmer / Cucchiarini, Catia / Kessens, Judith M. (2000): "Comparing the recognition performance of CSRs: in search of an adequate metric and statistical significance test", In ICSLP-2000, vol.4, 740-743.