Error Handling in Spoken Dialogue Systems August 28-31, 2003 Château d'Oex, Vaud, Switzerland

Process Discovery for Evaluating Dialogue Strategies

Piroska Lendvai, Laura Maruster

(1) ILK Research Group, Computational Linguistics & AI, Tilburg University, The Netherlands
(2) Faculty of Technology Management, Eindhoven University of Technology, The Netherlands

We develop a diagnostic method for detecting problematic dialogue strategies in spoken dialogue systems (SDS). Our tool classifies relations between semantically-pragmatically labelled system prompts and user inputs in a Dutch SDS corpus, and constructs a dependency/frequency graph that provides rich information about the underlying interaction process. The graph can be directly analysed to specify those system events from which more paths lead to problematic user events than to the unproblematic semantic equivalents of those user inputs. As a result, we can pinpoint bad sequences of prompting strategies in the system and suggest ways to replace such prompts with more effective ones. Our approach is a general method that provides straightforward output for evaluating a SDS design, given some labelled data.

Full Paper

Bibliographic reference.  Lendvai, Piroska / Maruster, Laura (2003): "Process discovery for evaluating dialogue strategies", In EHSD-2003, 119-122.