International Workshop on Spoken Language Translation (IWSLT) 2012

Hong Kong
December 6-7, 2012

Overview of the IWSLT 2012 Evaluation Campaign

Marcello Federico (1), Mauro Cettolo (1), Luisa Bentivogli (2), Michael Paul (3), Sebastian Stüker

(1) FBK Povo (Trento), Italy
(2) CELCT, Povo (Trento), Italy
(3) NICT, Hikaridai, Kyoto, Japan
(4) KIT, Karlsruhe, Germany

We report on the ninth evaluation campaign organized by the IWSLT workshop. This year, the evaluation offered multi- ple tracks on lecture translation based on the TED corpus, and one track on dialog translation from Chinese to English based on the Olympic trilingual corpus. In particular, the TED tracks included a speech transcription track in English, a speech translation track from English to French, and text translation tracks from English to French and from Arabic to English. In addition to the official tracks, ten unofficial MT tracks were offered that required translating TED talks into English from either Chinese, Dutch, German, Polish, Por- tuguese (Brazilian), Romanian, Russian, Slovak, Slovene, or Turkish. 16 teams participated in the evaluation and sub- mitted a total of 48 primary runs. All runs were evaluated with objective metrics, while runs of the official translation tracks were also ranked by crowd-sourced judges. In par- ticular, subjective ranking for the TED task was performed on a progress test which permitted direct comparison of the results from this year against the best results from the 2011 round of the evaluation campaign.

Full Paper    Presentation

Bibliographic reference.  Federico, Marcello / Cettolo, Mauro / Bentivogli, Luisa / Paul, Michael / Stüker, Sebastian (2012): "Overview of the IWSLT 2012 evaluation campaign", In IWSLT-2012, 12-33.