Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Use of Recursive Mumble Models for Confidence Measuring

Qiguang Lin, David Lubensky, Salim Roukos

IBM T.J. Watson Research Center, Computer Science Department, Yorktown Heights, NY, USA

In many speech recognition applications such as name dialing, it is necessary to have the ability to know when a recognition error has occurred so that undesired or unpredicted system behavior can be minimized. Confidence measure is usually used for detection of probable errors. In this paper, a new method for measuring confidence is presented. The method is based on use of recursive mumble models. During a regular decoding from which word hypotheses and word boundaries are known, the score of recursive mumble models is then determined. The (weighted) difference between the word detail-match score and the mumble score is used as the confidence measure. It is next compared to a predefined threshold to decide whether the decoded result is confidently correct or not. The method has been evaluated with two different databases. The results show that the new method outperforms our previous method solely based on the word detail-match scores. In particular, the results show that the new method is able to reduce the equal error rate from 32% to 23% and that it rejects far more (78% versus 35%) out-of-domain sentences at the fixed 5% false rejection rate.

Keywords: confidence measure, mumble model, equal error rate

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Lin, Qiguang / Lubensky, David / Roukos, Salim (1999): "Use of recursive mumble models for confidence measuring", In EUROSPEECH'99, 53-56.