8th Annual Conference of the International Speech Communication Association

Antwerp, Belgium
August 27-31, 2007

How Predictable is ASR Confidence in Dialog Applications?

Xiang Li, Juan M. Huerta

IBM T.J. Watson Research Center, USA

ASR confidence is a metric that reflects, to a large extent, the conditions under which a recognition task is being carried out as well as the reliability of the result. Because of this, ASR confidence constitutes a potentially useful feature in frameworks that attempt to asses the state of a dialog. In this paper we evaluate the predictability of ASR confidence based on knowledge of previously observed context-dependent confidences. We find out that the contextual confidence can be predicted with a standard prediction deviation less than 10% of the dynamic range of the confidence score, which represents a almost 40% relative reduction in standard deviation measure to a static confidence assumption baseline. Because our prediction is based on context, this predictability can be leveraged to produce an estimate of the expected average confidence until the end of a call based on the context path expected to be traversed.

Full Paper

Bibliographic reference.  Li, Xiang / Huerta, Juan M. (2007): "How predictable is ASR confidence in dialog applications?", In INTERSPEECH-2007, 1745-1748.