Sixth International Conference on Spoken Language Processing
In previous work, we have argued that it is beneficial to find confidence measures (CM’s) that are not dependent on use of "side information" from a specific recogniser. Here, we extend this philosophy to include the use of semantic information in estimating the confidence that a word is correct. We are motivated by the observation that sometimes the recogniser outputs a word which can easily be spotted (by humans) as incorrect, because it bears no relation to the semantics of the rest of the decoded sentence. Latent semantic analysis (LSA) was used as a method for estimating semantic "semantic similarity" between words in a text corpus. From these scores, an average semantic similarity of each decoded word to the other decoded words in an utterance could be estimated, and by thresholding this similarity measure, words were tagged as CORRECT or INCORRECT. We benchmarked the performance of this semantic CM against a tried-and-tested CM, the N-best CM. The precision of the semantic CM was inferior to that of N-best when the recall (the number of words considered) was high, but it out-performed N-best for low recall, and a combined classifier showed the benefits of using both techniques. An interesting and unexpected result was that the semantic CM was better at identifying correct words than incorrect words.
Bibliographic reference. Cox, Stephen / Dasmahapatra, Srinandan (2000): "A semantically-based confidence measure for speech recognition", In ICSLP-2000, vol.4, 206-209.