ISCA Archive Interspeech 2009
ISCA Archive Interspeech 2009

Predicting the quality of multimodal systems based on judgments of single modalities

Ina Wechsung, Klaus-Peter Engelbrecht, Anja B. Naumann, Stefan Schaffer, Julia Seebode, Florian Metze, Sebastian Möller

This paper investigates the relationship between user ratings of multimodal systems and user ratings of its single modalities. Based on previous research showing precise predictions of ratings of multimodal systems based on ratings of single modality, it was hypothesized that the accuracy might have been caused by the participants’ efforts to rate consistently. We address this issue with two new studies. In the first study, the multimodal system was presented before the single modality versions were known by the users. In the second study, the type of system was changed, and age effects were investigated. We apply linear regression and show that models get worse when the order is changed. In addition, models for younger users perform better than those for older users. We conclude that ratings can be impacted by the effort of users to judge consistently, as well as their ability to do so.


doi: 10.21437/Interspeech.2009-532

Cite as: Wechsung, I., Engelbrecht, K.-P., Naumann, A.B., Schaffer, S., Seebode, J., Metze, F., Möller, S. (2009) Predicting the quality of multimodal systems based on judgments of single modalities. Proc. Interspeech 2009, 1827-1830, doi: 10.21437/Interspeech.2009-532

@inproceedings{wechsung09_interspeech,
  author={Ina Wechsung and Klaus-Peter Engelbrecht and Anja B. Naumann and Stefan Schaffer and Julia Seebode and Florian Metze and Sebastian Möller},
  title={{Predicting the quality of multimodal systems based on judgments of single modalities}},
  year=2009,
  booktitle={Proc. Interspeech 2009},
  pages={1827--1830},
  doi={10.21437/Interspeech.2009-532}
}