INTERSPEECH 2004 - ICSLP
This paper discusses efficient parameter estimation methods for joint (unconditional) maximum entropy language models such as whole-sentence models. Such models are a sound framework for formalizing arbitrary linguistic knowledge in a consistent manner. It has been shown that general-purpose gradient-based optimization methods are among the most efficient algorithms for parameter estimation for several tasks in natural language processing. This paper applies gradient methods to whole-sentence language models and other domains whose sample spaces are infinite or practically innumerable and require simulation. It also presents Open Source software for easily fitting and testing joint maximum entropy models.
Bibliographic reference. Schofield, Edward James (2004): "Fast parameter estimation for joint maximum entropy language models", In INTERSPEECH-2004, 2241-2244.