Sixth European Conference on Speech Communication and Technology
(EUROSPEECH'99)

Budapest, Hungary
September 5-9, 1999

Evaluation of Confidence Measures for Language Identification

Kay Berkling, Douglas A. Reynolds, Marc Zissman

M.I.T. Lincoln Laboratory, Lexington, MA, USA

In this paper we examine various ways to derive confidence measures for a language identification system [3], using phone recognition followed by language models, and describe the application of an evaluation metric [1] for measuring the “goodness" of the different confidence measures. Experiments are conducted on the 1996 NIST Language Identification Evaluation corpus (derived from the Callfriend corpus of conversational telephone speech). The system is trained on the NIST 96 development data and evaluated on the NIST 96 evaluation data. Results indicate that we are able to predict the performance of a system and quantitatively evaluate how well the prediction holds on new data.


Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Berkling, Kay / Reynolds, Douglas A. / Zissman, Marc (1999): "Evaluation of confidence measures for language identification", In EUROSPEECH'99, 363-366.