EUROSPEECH 2003 - INTERSPEECH 2003 8th European Conference on Speech Communication and Technology Geneva, Switzerland September 1-4, 2003

We are not Amused - But How do You Know? User States in a Multi-Modal Dialogue System

Anton Batliner, Viktor Zeissler, Carmen Frank, Johann Adelhardt, Rui P. Shi, Elmar Nöth

Universität Erlangen-Nürnberg, Germany

For the multi-modal dialogue system SmartKom, emotional user states in a Wizard-of-Oz experiment as, e.g., joyful, angry, helpless, are annotated holistically and based purely on facial expressions; other phenomena (prosodic peculiarities, offtalk, i.e., speaking aside, etc.) are labelled as well. We present the correlations between these different annotations and report classification results using a large prosodic feature vector. The performance of the user state classification is not yet satisfactory; possible reasons and remedies are discussed.

Full Paper

Bibliographic reference.  Batliner, Anton / Zeissler, Viktor / Frank, Carmen / Adelhardt, Johann / Shi, Rui P. / Nöth, Elmar (2003): "We are not amused - but how do you know? user states in a multi-modal dialogue system", In EUROSPEECH-2003, 733-736.