Interspeech'2005 - Eurospeech

Lisbon, Portugal
September 4-8, 2005

A Methodology for Comparing Grammar-Based and Robust Approaches to Speech Understanding

Manny Rayner (1), Pierrette Bouillon (1), Nikos Chatzichrisafis (1), Beth Ann Hockey (2), Marianne Santaholma (1), Marianne Starlander (1), Hitoshi Isahara (3), Kyoko Kanzaki (3), Yukie Nakao (3)

(1) University of Geneva, Switzerland; (2) University of California at Santa Cruz, USA; (3) NICT, Japan

We present a series of experiments designed to compare grammarbased and robust approaches to speech understanding, performed in the context of an Open Source medical speech translation system. We used two versions of the system, one grammar-based and one robust, trained off the same training data, and evaluated them on test data collected using both versions of the system. The experiments were constructed so as to avoid several methodological problems which occurred in earlier work reported in the literature. We found that the grammar-based version gave significantly better results than the robust version, with the difference increasing as subjects became more familiar with the system's coverage. The rate of improvement in subject performance was positively affected by providing them with an intelligent online help system.

Full Paper

Bibliographic reference.  Rayner, Manny / Bouillon, Pierrette / Chatzichrisafis, Nikos / Hockey, Beth Ann / Santaholma, Marianne / Starlander, Marianne / Isahara, Hitoshi / Kanzaki, Kyoko / Nakao, Yukie (2005): "A methodology for comparing grammar-based and robust approaches to speech understanding", In INTERSPEECH-2005, 1877-1880.