Interspeech'2005 - Eurospeech

Lisbon, Portugal
September 4-8, 2005

Language Model Data Filtering via User Simulation and Dialogue Resynthesis

Chao Wang (1), Stephanie Seneff (1), Grace Chung (2)

(1) Massachusetts Institute of Technology, USA; (2) Corporation for National Research Initiatives, USA

In this paper, we address the issue of generating language model training data during the initial stages of dialogue system development. The process begins with a large set of sentence templates, automatically adapted from other application domains. We propose two methods to filter the raw data set to achieve a desired probability distribution of the semantic content, both on the sentence level and on the class level. The first method utilizes user simulation technology, which obtains the probability model via an interplay between a probabilistic user model and the dialogue system. The second method synthesizes novel dialogue interactions by modeling after a small set of dialogues produced by the developers during the course of system refinement. We evaluated our methodology by speech recognition performance on a set of 520 unseen utterances from naive users interacting with a restaurant domain dialogue system.

Full Paper

Bibliographic reference.  Wang, Chao / Seneff, Stephanie / Chung, Grace (2005): "Language model data filtering via user simulation and dialogue resynthesis", In INTERSPEECH-2005, 21-24.