10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

A User Modeling-Based Performance Analysis of a Wizarded Uncertainty-Adaptive Dialogue System Corpus

Kate Forbes-Riley, Diane Litman

University of Pittsburgh, USA

Motivated by prior spoken dialogue system research in user modeling, we analyze interactions between performance and user class in a dataset previously collected with two wizarded spoken dialogue tutoring systems that adapt to user uncertainty. We focus on user classes defined by expertise level and gender, and on both objective (learning) and subjective (user satisfaction) performance metrics. We find that lower expertise users learn best from one adaptive system but prefer the other, while higher expertise users learned more from one adaptive system but didnít prefer either. Female users both learn best from and prefer the same adaptive system, while males preferred one adaptive system but didnít learn more from either. Our results yield an empirical basis for future investigations into whether adaptive system performance can improve by adapting to user uncertainty differently based on user class.

Full Paper

Bibliographic reference.  Forbes-Riley, Kate / Litman, Diane (2009): "A user modeling-based performance analysis of a wizarded uncertainty-adaptive dialogue system corpus", In INTERSPEECH-2009, 2467-2470.