ISCA Archive Interspeech 2009
ISCA Archive Interspeech 2009

Influence of training on direct and indirect measures for the evaluation of multimodal systems

Julia Seebode, Stefan Schaffer, Ina Wechsung, Florian Metze

Finding suitable evaluation methods is an indispensable task during the development of new user interfaces, as no standardized approach has so far been established, especially for multimodal interfaces. In the current study, we used several data sources (direct and indirect measurements) to evaluate a multimodal version of an information system, tested on trained and untrained users. We investigated the extent to which the different types of data showed concordance concerning the perceived quality of the system, in order to derive clues as to the suitability of the respective evaluation methods. The aim was to examine, if widely used methods not originally developed for multimodal interfaces are appropriate under these conditions, and to derive new evaluation paradigms.


doi: 10.21437/Interspeech.2009-100

Cite as: Seebode, J., Schaffer, S., Wechsung, I., Metze, F. (2009) Influence of training on direct and indirect measures for the evaluation of multimodal systems. Proc. Interspeech 2009, 300-303, doi: 10.21437/Interspeech.2009-100

@inproceedings{seebode09_interspeech,
  author={Julia Seebode and Stefan Schaffer and Ina Wechsung and Florian Metze},
  title={{Influence of training on direct and indirect measures for the evaluation of multimodal systems}},
  year=2009,
  booktitle={Proc. Interspeech 2009},
  pages={300--303},
  doi={10.21437/Interspeech.2009-100}
}