11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Say It As You Mean It - Analyzing Free User Comments in the VOICE Awards Corpus

Florian Gödde, Sebastian Möller

Deutsche Telekom Laboratories, Germany

Usability questionnaires usually contain scales related to effectiveness, efficiency and overall satisfaction which provide a quantitative value for the user’s opinion. However, analyzing quantitative data often does not show the reason underlying for a good or bad opinion. Simple questions like “What did you like about the system?” and “What did you not like about the system?” can shade light on the underlying reasons, but a lot of effort is needed for the analysis of such data. Nevertheless, the answers to these questions contain the users’ opinion in their own words and hence often show high correlation with the overall rating of the system. In the frame of the SpeechEval project we analyzed the German VOICE Awards corpus over three consecutive years, categorizing the answers to these two free text questions and analyzing correlations between the categories and the overall rating of the systems.

Full Paper

Bibliographic reference.  Gödde, Florian / Möller, Sebastian (2010): "Say it as you mean it - analyzing free user comments in the VOICE awards corpus", In INTERSPEECH-2010, 1978-1981.