Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Knowledge Collection for Natural Language Spoken Dialog Systems

Egbert Ammicht (1), Allen Gorin, Tirso Alonso

AT&T Laboratories Research Florham Park, NJ, USA
(2) Now at Bell Laboratories, Lucent Technologies, Whippany, NJ, USA

The development of natural language spoken dialog systems requires collection and labeling of a large set of user-system interactions (knowledge). Ideally, it should be collected from live traffic in the field, since scripted scenarios in a lab typically result in unnatural phraseology. To achieve this, we introduce an extension of the "Wizard of Oz", a hidden human agent who oversees the machine side of the interaction in collaboration with the automated dialog manager, unbeknownst to the user. The Wizard can provide a continuum of supervision ranging from explicitly controlling every step to a Wizard-override mode where the machine operates semi-autonomously and the human overrides only when necessary. All interactions are instrumented and entered directly into a database using a standard interface and SQL. The collected data will typically be analyzed in the laboratory to quantify system performance, and to introduce algorithmic improvements prior to a new set of experiments. Over the course of system development iterations, the nature of the Wizard evolves over a continuum of functions from actual control (in the absence of any system knowledge) to passive observer. The user cannot tell the difference.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Ammicht, Egbert / Gorin, Allen / Alonso, Tirso (1999): "Knowledge collection for natural language spoken dialog systems", In EUROSPEECH'99, 1375-1378.