International Workshop on Spoken Language Translation (IWSLT) 2006

Keihanna Science City, Kyoto, Japan
November 27-28, 2006

IWSLT-06: Experiments with Commercial MT Systems and Lessons from Subjective Evaluations

Christian Boitet, Youcef Bey, Mutsuko Tomokiyo, Wenjie Cao, Hervé Blanchon

GETA, laboratoire CLIPS, IMAG-campus, Grenoble, France

This is a short report of our participation to IWSLT-06. First, we let 2 commercial systems participate as fairly as possible (SYSTRAN v5.0 for CE, JE, AE, & IE, Atlas-II for JE), taking care of preprocessing and postprocessing tasks, and tuning as many "pairs" as possible by creating "user dictionaries" and finding a good combination of parameters (such as dictionary priority). Second, we took part in the subjective evaluation of CE results (fluency and adequacy). Details on experiments and methodological remarks are provided, with a perspective to introduce less expensive and more objective human- and task-related evaluation methods.

Full Paper     Presentation

Bibliographic reference.  Boitet, Christian / Bey, Youcef / Tomokiyo, Mutsuko / Cao, Wenjie / Blanchon, Hervé (2006): "IWSLT-06: experiments with commercial MT systems and lessons from subjective evaluations", In IWSLT-2006, 23-30.