8th International Conference on Spoken Language Processing

Jeju Island, Korea
October 4-8, 2004

Vocabulary and Language Model Adaptation using Information Retrieval

Brigitte Bigi (1), Yan Huang (1), Renato De Mori (2)

(1) University of California, Berkeley, USA
(2) Université d'Avignon, France

The goal of vocabulary optimization is to construct a vocabulary with exactly those words that are the most likely to appear in the test data. We will present a new approach to reduce the out-of-vocabulary (OOV) rate by adapting the vocabulary model during the ASR process. This method can also be used for the statistial language model (SLM) adaptation. An information retrieval system is used after the first pass of the ASR system to obtain a set of relevant documents. These documents are then used to generate the new vocabulary and/or corpus. In this paper, we propose a new retrieving method well-adapted for this purpose. Experiments were carried out on French with a 28% OOV rate reduction. Experiments were also carried out on English for the SLM adaptation, with 7.9% perplexity reduction, and minor WER improvement.

Full Paper

Bibliographic reference.  Bigi, Brigitte / Huang, Yan / Mori, Renato De (2004): "Vocabulary and language model adaptation using information retrieval", In INTERSPEECH-2004, 1361-1364.