12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

User Simulation in Dialogue Systems Using Inverse Reinforcement Learning

Senthilkumar Chandramohan (1), Matthieu Geist (1), Fabrice Lefèvre (2), Olivier Pietquin (1)

(1) Supélec, France
(2) LIA, France

Spoken Dialogue Systems (SDS) are man-machine interfaces which use natural language as the medium of interaction. Dialogue corpora collection for the purpose of training and evaluating dialogue systems is an expensive process. User simulators aim at simulating human users in order to generate synthetic data. Existing methods for user simulation mainly focus on generating data with the same statistical consistency as in some reference dialogue corpus. This paper outlines a novel approach for user simulation based on Inverse Reinforcement Learning (IRL). The task of building the user simulator is perceived as a task of imitation learning.

Full Paper

Bibliographic reference.  Chandramohan, Senthilkumar / Geist, Matthieu / Lefèvre, Fabrice / Pietquin, Olivier (2011): "User simulation in dialogue systems using inverse reinforcement learning", In INTERSPEECH-2011, 1025-1028.