Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Towards Improved Language Model Evaluation Measures

Philip Clarkson, Tony Robinson

Cambridge University Engineering Department, Cambridge, UK

Much recent research has demonstrated that the correlation between a language modelís perplexity and its effect on the word error rate of a speech recognition system is not as strong as was once thought. This represents a major problem for those in-volved in developing language models. This paper describes the development of new measures of language model quality. These measures retain the ease of computation and task inde-pendence that are perplexityís strengths, yet are considerably better correlated with word error rate. This paper also shows that mixture-based language models are improved by applying interpolation weights which are optimised with respect to these new measures, rather than a maximum likelihood criterion.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Clarkson, Philip / Robinson, Tony (1999): "Towards improved language model evaluation measures", In EUROSPEECH'99, 1927-1930.