INTERSPEECH 2004 - ICSLP 8th International Conference on Spoken Language Processing Jeju Island, Korea October 4-8, 2004

Fast Parameter Estimation for Joint Maximum Entropy Language Models

Edward James Schofield

Imperial College London, Austria

This paper discusses efficient parameter estimation methods for joint (unconditional) maximum entropy language models such as whole-sentence models. Such models are a sound framework for formalizing arbitrary linguistic knowledge in a consistent manner. It has been shown that general-purpose gradient-based optimization methods are among the most efficient algorithms for parameter estimation for several tasks in natural language processing. This paper applies gradient methods to whole-sentence language models and other domains whose sample spaces are infinite or practically innumerable and require simulation. It also presents Open Source software for easily fitting and testing joint maximum entropy models.

Full Paper

Bibliographic reference.  Schofield, Edward James (2004): "Fast parameter estimation for joint maximum entropy language models", In INTERSPEECH-2004, 2241-2244.