ISCA Archive Eurospeech 1999
ISCA Archive Eurospeech 1999

Towards improved language model evaluation measures

Philip Clarkson, Tony Robinson

Much recent research has demonstrated that the correlation between a language model’s perplexity and its effect on the word error rate of a speech recognition system is not as strong as was once thought. This represents a major problem for those in-volved in developing language models. This paper describes the development of new measures of language model quality. These measures retain the ease of computation and task inde-pendence that are perplexity’s strengths, yet are considerably better correlated with word error rate. This paper also shows that mixture-based language models are improved by applying interpolation weights which are optimised with respect to these new measures, rather than a maximum likelihood criterion.


doi: 10.21437/Eurospeech.1999-423

Cite as: Clarkson, P., Robinson, T. (1999) Towards improved language model evaluation measures. Proc. 6th European Conference on Speech Communication and Technology (Eurospeech 1999), 1927-1930, doi: 10.21437/Eurospeech.1999-423

@inproceedings{clarkson99_eurospeech,
  author={Philip Clarkson and Tony Robinson},
  title={{Towards improved language model evaluation measures}},
  year=1999,
  booktitle={Proc. 6th European Conference on Speech Communication and Technology (Eurospeech 1999)},
  pages={1927--1930},
  doi={10.21437/Eurospeech.1999-423}
}