ISCA Archive SLTU 2014
ISCA Archive SLTU 2014

Sequence memoizer based language model for Russian speech recognition

Daria Vazhenina, Konstantin Markov

In this paper, we propose a novel language model for Russian large vocabulary speech recognition based on sequence memoizer modeling technique. Sequence memoizer is a long span text dependency model and was initially proposed for character language modeling. Here, we use it to build word level language model (LM) in ASR. We compare its performance with recurrent neural network (RNN) LM, which also models long span word dependencies. A number of experiments were carried out using various amounts of train data and different text data arrangements. According to our experimental results, the sequence memoizer LM outperforms recurrent neural network and standard 3-gram LMs in terms of perplexity, while RNN LM achieves better word error rate. The lowest word error rate is achieved by combining all three language models together using linear interpolation.

Index Terms: sequence memoizer, advanced language modeling, inflective languages


Cite as: Vazhenina, D., Markov, K. (2014) Sequence memoizer based language model for Russian speech recognition. Proc. 4th Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU 2014), 183-187

@inproceedings{vazhenina14_sltu,
  author={Daria Vazhenina and Konstantin Markov},
  title={{Sequence memoizer based language model for Russian speech recognition}},
  year=2014,
  booktitle={Proc. 4th Workshop on Spoken Language Technologies for Under-Resourced Languages  (SLTU 2014)},
  pages={183--187}
}