ASR2000 - Automatic Speech Recognition: Challenges for the new Millenium September 18-20, 2000 Paris, France

Recognition Confidence Scoring for Use in Speech Understanding Systems

Timothy J. Hazen, Theresa Burianek, Joseph Polifroni, and Stephanie Seneff

Spoken Language Systems Group, MIT Laboratory for Computer Science, Cambridge, MA, USA

In this paper we present an approach to recognition confidence scoring and a method for integrating confidence scores into the understanding and dialogue components of a speech understanding system. The system uses a multi-tiered approach where con- fidence scores are computed at the phonetic, word, and utterance levels. The scores are produced by extracting confidence features from the computation of the recognition hypotheses and processing these features using an accept/reject classifier for word and utterance hypotheses. The output of the confidence classifiers can then be incorporated into the parsing mechanism of the language understanding component. To evaluate the system, experiments were conducted using the JUPITER weather information system. Evaluation was performed at the understanding level using key-value pair concept error rate as the evaluation metric. When confidence scores were integrated into the understanding component of the system, the concept error rate was reduced by over 35%.

Full Paper (PDF)   Full Paper (Zipped Postscript)

Bibliographic reference.  Hazen, Timothy J. / Burianek, Theresa / Polifroni, Joseph / Seneff, Stephanie (2000): "Recognition confidence scoring for use in speech understanding systems", In ASR-2000, 213-220.