Finding suitable evaluation methods is an indispensable task during the development of new user interfaces, as no standardized approach has so far been established, especially for multimodal interfaces. In the current study, we used several data sources (direct and indirect measurements) to evaluate a multimodal version of an information system, tested on trained and untrained users. We investigated the extent to which the different types of data showed concordance concerning the perceived quality of the system, in order to derive clues as to the suitability of the respective evaluation methods. The aim was to examine, if widely used methods not originally developed for multimodal interfaces are appropriate under these conditions, and to derive new evaluation paradigms.
Cite as: Seebode, J., Schaffer, S., Wechsung, I., Metze, F. (2009) Influence of training on direct and indirect measures for the evaluation of multimodal systems. Proc. Interspeech 2009, 300-303, doi: 10.21437/Interspeech.2009-100
@inproceedings{seebode09_interspeech,
author={Julia Seebode and Stefan Schaffer and Ina Wechsung and Florian Metze},
title={{Influence of training on direct and indirect measures for the evaluation of multimodal systems}},
year=2009,
booktitle={Proc. Interspeech 2009},
pages={300--303},
doi={10.21437/Interspeech.2009-100}
}