8th European Conference on Speech Communication and Technology

Geneva, Switzerland
September 1-4, 2003


Improving a Connectionist Based Syntactical Language Model

Ahmad Emami

Johns Hopkins University, USA

Using a connectionist model as one of the components of the Structured Language Model has lead to significant improvements in perplexity and word error rate, mainly because of the connectionist model's power in using longer contexts and its ability in fighting the data sparseness problem. For its training, the SLM needs the syntactical parses of the word strings in the training data, provided by either humans or an external parser. In this paper we study the effect of training the connectionist based language model on the hidden parses hypothesized by the SLM itself. Since multiple partial parses are constructed for each word position, the model and the log-likelihood function will be in a form that necessitates a specific manner of training of the connectionist model. Experiments on the UPENN section of the Wall Street Journal corpus show significant improvements in perplexity.

Full Paper

Bibliographic reference.  Emami, Ahmad (2003): "Improving a connectionist based syntactical language model", In EUROSPEECH-2003, 413-416.