International Workshop on Spoken Language Translation (IWSLT) 2010

Paris, France
December 2-3, 2010

Comparing Intrinsic and Extrinsic Evaluation of MT Output in a Dialogue System

Anne H. Schneider, Ielka van der Sluis, Saturnino Luz

Department of Computer Science, Trinity College Dublin, Ireland

We present an exploratory study to assess machine translation output for application in a dialogue system using an intrinsic and an extrinsic evaluation method. For the intrinsic evaluation we developed an annotation scheme to determine the quality of the translated utterances in isolation. For the extrinsic evaluation we employed theWizard of Oz technique to assess the quality of the translations in the context of a dialogue application. Results differ and we discuss the possible reasons for this outcome.

Full Paper

Bibliographic reference.  Schneider, Anne H. / Sluis, Ielka van der / Luz, Saturnino (2010): "Comparing intrinsic and extrinsic evaluation of MT output in a dialogue system", In IWSLT-2010, 329-336.