EUROSPEECH 2003 - INTERSPEECH 2003
Using a connectionist model as one of the components of the Structured Language Model has lead to significant improvements in perplexity and word error rate, mainly because of the connectionist model's power in using longer contexts and its ability in fighting the data sparseness problem. For its training, the SLM needs the syntactical parses of the word strings in the training data, provided by either humans or an external parser. In this paper we study the effect of training the connectionist based language model on the hidden parses hypothesized by the SLM itself. Since multiple partial parses are constructed for each word position, the model and the log-likelihood function will be in a form that necessitates a specific manner of training of the connectionist model. Experiments on the UPENN section of the Wall Street Journal corpus show significant improvements in perplexity.
Bibliographic reference. Emami, Ahmad (2003): "Improving a connectionist based syntactical language model", In EUROSPEECH-2003, 413-416.