International Workshop on Spoken Language Translation (IWSLT) 2004

Keihanna Science City, Kyoto, Japan
September 30-October 1, 2004

Towards Fairer Evaluations of Commercial MT Systems on Basic Travel Expressions Corpora

Hervé Blanchon (1), Christian Boitet (1), Francis Brunet-Manquat (1), Mutsuko Tomokiyo (1), Agnès Hamon (2), Vo Trung Hung (1), Youcef Bey (1)

(1) Laboratoire CLIPS, Grenoble, France
(2) Laboratoire de Statistique (Université Haute Bretagne), Rennes, France

We compare the performance of several SYSTRAN systems on the BTEC corpus. Two language pairs: Chinese to English and Japanese to English are used. Whenever it is possible the system will be used “off the shelf” and then tuned.
   The first system we use is freely available on the web. The second system, SYSTRAN Premium, is commercial. It is used in two ways: (1) choosing and ordering available original dictionaries and setting parameters, (2) same + user dictionaries.
   As far as the evaluation is concerned, we competed in the unlimited data track.

Full Paper    Presentation

Bibliographic reference.  Blanchon, Hervé / Boitet, Christian / Brunet-Manquat, Francis / Tomokiyo, Mutsuko / Hamon, Agnès / Hung, Vo Trung / Bey, Youcef (2004): "Towards fairer evaluations of commercial MT systems on basic travel expressions corpora", In IWSLT-2004, 21-26.