4th International Conference on Spoken Language Processing

Philadelphia, PA, USA
October 3-6, 1996

Modeling Long Distance Dependence in Language: Topic Mixtures vs. Dynamic Cache Models

R. Iyer, Mari Ostendorf

Electrical and Computer Engineering Department, Boston University, Boston, MA, USA

In this paper,we investigate a new statistical language model which captures topic-related dependencies of words within and across sentences. First, we develop a sentence-level mixture language model that takes advantage of the topic constraints in a sentence or article. Second, we introduce topic-dependent dynamic cache adaptation techniques in the framework of the mixture model. Experiments with the static (or unadapted) mixture model on the 1994 WSJ task indicated a 21% reduction in perplexity and a 3-4% improvement in recognition accuracy over a general n-gram model. The static mixture model also improved recognition performance over an adapted n-gram model. Mixture adaptation techniques contributed a further 14%reduction in perplexity and a small improvement in recognition accuracy.

Full Paper

Bibliographic reference.  Iyer, R. / Ostendorf, Mari (1996): "Modeling long distance dependence in language: topic mixtures vs. dynamic cache models", In ICSLP-1996, 236-239.