International Workshop on Spoken Language Translation (IWSLT) 2005

Pittsburgh, PA, USA
October 24-25, 2005

Evaluating Machine Translation Output with Automatic Sentence Segmentation

Evgeny Matusov, Gregor Leusch, Oliver Bender, Hermann Ney

Lehrstuhl für Informatik VI - Computer Science Department, RWTH Aachen University, Aachen, Germany

This paper presents a novel automatic sentence segmentation method for evaluating machine translation output with possibly erroneous sentence boundaries. The algorithm can process translation hypotheses with segment boundaries which do not correspond to the reference segment boundaries, or a completely unsegmented text stream. Thus, the method is especially useful for evaluating translations of spoken language. The evaluation procedure takes advantage of the edit distance algorithm and is able to handle multiple reference translations. It efficiently produces an optimal automatic segmentation of the hypotheses and thus allows application of existing well-established evaluation measures. Experiments show that the evaluation measures based on the automatically produced segmentation correlate with the human judgement at least as well as the evaluation measures which are based on manual sentence boundaries.

Full Paper    Presentation

Bibliographic reference.  Matusov, Evgeny / Leusch, Gregor / Bender, Oliver / Ney, Hermann (2005): "Evaluating machine translation output with automatic sentence segmentation", In IWSLT-2005, 138-144.