11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Evaluating a Dialog Language Generation System: Comparing the Mountain System to Other NLG Approaches

Brian Langner, Stephan Vogel, Alan W. Black

Carnegie Mellon University, USA

This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.

Full Paper

Bibliographic reference.  Langner, Brian / Vogel, Stephan / Black, Alan W. (2010): "Evaluating a dialog language generation system: comparing the mountain system to other NLG approaches", In INTERSPEECH-2010, 1109-1112.