Sequence to Sequence Modeling for User Simulation in Dialog Systems

Paul Crook, Alex Marin


User simulators are a principal offline method for training and evaluating human-computer dialog systems. In this paper, we examine simple sequence-to-sequence neural network architectures for training end-to-end, natural language to natural language, user simulators, using only raw logs of previous interactions without any additional human labelling. We compare the neural network-based simulators with a language model (LM)-based approach for creating natural language user simulators. Using both an automatic evaluation using LM perplexity and a human evaluation, we demonstrate that the sequence-to-sequence approaches outperform the LM-based method. We show correlation between LM perplexity and the human evaluation on this task, and discuss the benefits of different neural network architecture variations.


 DOI: 10.21437/Interspeech.2017-161

Cite as: Crook, P., Marin, A. (2017) Sequence to Sequence Modeling for User Simulation in Dialog Systems. Proc. Interspeech 2017, 1706-1710, DOI: 10.21437/Interspeech.2017-161.


@inproceedings{Crook2017,
  author={Paul Crook and Alex Marin},
  title={Sequence to Sequence Modeling for User Simulation in Dialog Systems},
  year=2017,
  booktitle={Proc. Interspeech 2017},
  pages={1706--1710},
  doi={10.21437/Interspeech.2017-161},
  url={http://dx.doi.org/10.21437/Interspeech.2017-161}
}