User simulators are a principal offline method for training and evaluating human-computer dialog systems. In this paper, we examine simple sequence-to-sequence neural network architectures for training end-to-end, natural language to natural language, user simulators, using only raw logs of previous interactions without any additional human labelling. We compare the neural network-based simulators with a language model (LM)-based approach for creating natural language user simulators. Using both an automatic evaluation using LM perplexity and a human evaluation, we demonstrate that the sequence-to-sequence approaches outperform the LM-based method. We show correlation between LM perplexity and the human evaluation on this task, and discuss the benefits of different neural network architecture variations.
Cite as: Crook, P., Marin, A. (2017) Sequence to Sequence Modeling for User Simulation in Dialog Systems. Proc. Interspeech 2017, 1706-1710, doi: 10.21437/Interspeech.2017-161
@inproceedings{crook17_interspeech,
author={Paul Crook and Alex Marin},
title={{Sequence to Sequence Modeling for User Simulation in Dialog Systems}},
year=2017,
booktitle={Proc. Interspeech 2017},
pages={1706--1710},
doi={10.21437/Interspeech.2017-161}
}