International Workshop on Spoken Language Translation (IWSLT) 2005

Pittsburgh, PA, USA
October 24-25, 2005

Overview of the IWSLT 2005 Evaluation Campaign

Matthias Eck, Chiori Hori

Interactive Systems Laboratories, Carnegie Mellon University, Pittsburgh, PA, USA

This paper reports an overview of the evaluation campaign results of the IWSLT 2005 workshop1. The BTEC corpus, which consists of typical travel domain phrases, was used. Data for the five language pairs Arabic/Chinese/Japanese/Korean to English and English to Chinese was prepared. To study how much the amount of the training data and how much different training and decoding approaches contribute to the performance, a supplied data and an unrestricted data track were introduced. In addition, translation results were evaluated not only for text input but also speech recognition output. 19 systems from 17 organizations participated in the evaluation. All machine translation results were evaluated using automatic evaluation metrics. The most popular track, translating text form Chinese to English, was graded by 3 humans in terms of Fluency, Adequacy and Meaning Maintenance. The correlation between automatic evaluation metrics and human judgment was examined.

Full Paper    Presentation

Bibliographic reference.  Eck, Matthias / Hori, Chiori (2005): "Overview of the IWSLT 2005 evaluation campaign", In IWSLT-2005, 1-22.