ISCA Archive Interspeech 2005
ISCA Archive Interspeech 2005

Learning user simulations for information state update dialogue systems

Kallirroi Georgila, James Henderson, Oliver Lemon

This paper describes and compares two methods for simulating user behaviour in spoken dialogue systems. User simulations are important for automatic dialogue strategy learning and the evaluation of competing strategies. Our methods are designed for use with "Information State Update" (ISU)-based dialogue systems. The first method is based on supervised learning using linear feature combination and a normalised exponential output function. The user is modelled as a stochastic process which selects user actions ({ speech act, task } pairs) based on features of the current dialogue state, which encodes the whole history of the dialogue. The second method uses n-grams of { speech act, task } pairs, restricting the length of the history considered by the order of the n-gram. Both models were trained and evaluated on a subset of the Communicator corpus, to which we added annotations for user actions and Information States. The model based on linear feature combination has a perplexity of 2.08 whereas the best n-gram (4-gram) has a perplexity of 3.58. Each one of the user models ran against a system policy trained on the same corpus with a method similar to the one used for our linear feature combination model. The quality of the simulated dialogues produced was then measured as a function of the filled slots, confirmed slots, and number of actions performed by the system in each dialogue. In this experiment both the linear feature combination model and the best n-grams (5-gram and 4-gram) produced similar quality simulated dialogues.

doi: 10.21437/Interspeech.2005-401

Cite as: Georgila, K., Henderson, J., Lemon, O. (2005) Learning user simulations for information state update dialogue systems. Proc. Interspeech 2005, 893-896, doi: 10.21437/Interspeech.2005-401

  author={Kallirroi Georgila and James Henderson and Oliver Lemon},
  title={{Learning user simulations for information state update dialogue systems}},
  booktitle={Proc. Interspeech 2005},