5th International Conference on Spoken Language Processing Sydney, Australia November 30 - December 4, 1998

Confidence Scoring for Speech Understanding Systems

Christine Pao, Philipp Schmid, James R. Glass

MIT Lab for Computer Science, USA

This research investigates the use of utterance-level features for confidence scoring. Confidence scores are used to accept or reject user utterances in our conversational weather information system. We have developed an automatic labeling algorithm based on a semantic frame comparison between recognized and transcribed orthographies. We explore recognition-based features along with semantic, linguistic, and application-specific features for utterance rejection. Discriminant analysis is used in an iterative process to select the best set of classification features for our utterance rejection sub-system. Experiments show that we can correctly reject over 60% of incorrectly understood utterances while accepting 98% of all correctly understood utterances.

Full Paper

Bibliographic reference.  Pao, Christine / Schmid, Philipp / Glass, James R. (1998): "Confidence scoring for speech understanding systems", In ICSLP-1998, paper 0392.