ISCA Archive Interspeech 2007
ISCA Archive Interspeech 2007

A method for evaluating task-oriented spoken dialog translation systems based on communication efficiency

Toshiyuki Takezawa, Masahide Mizushima, Tohru Shimizu, Genichiro Kikui

We propose a method for measuring communication efficiency from the viewpoint of conveying essential information in task-oriented spoken dialog translation. We present the results of one dialog experiment using speech-to-speech translation systems and a similar experiment using the Wizard of Oz method, which was carried out using hidden interpreters instead of a speech-to-speech translation system. We also present the relative performance score of the speech-to-speech translation system, which was obtained by comparing the machine's performance with that of humans, i.e., hidden interpreters. Finally, we discuss the relationship between users' linguistic behavior and system performance. We found that users of the system tended to make shorter utterances without decreasing the number of essential items needed to achieve a task and to improve transmission efficiency by using strategy to control dialogs.


doi: 10.21437/Interspeech.2007-410

Cite as: Takezawa, T., Mizushima, M., Shimizu, T., Kikui, G. (2007) A method for evaluating task-oriented spoken dialog translation systems based on communication efficiency. Proc. Interspeech 2007, 1342-1345, doi: 10.21437/Interspeech.2007-410

@inproceedings{takezawa07_interspeech,
  author={Toshiyuki Takezawa and Masahide Mizushima and Tohru Shimizu and Genichiro Kikui},
  title={{A method for evaluating task-oriented spoken dialog translation systems based on communication efficiency}},
  year=2007,
  booktitle={Proc. Interspeech 2007},
  pages={1342--1345},
  doi={10.21437/Interspeech.2007-410}
}