The development of natural language spoken dialog systems requires collection and labeling of a large set of user-system interactions (knowledge). Ideally, it should be collected from live traffic in the field, since scripted scenarios in a lab typically result in unnatural phraseology. To achieve this, we introduce an extension of the "Wizard of Oz", a hidden human agent who oversees the machine side of the interaction in collaboration with the automated dialog manager, unbeknownst to the user. The Wizard can provide a continuum of supervision ranging from explicitly controlling every step to a Wizard-override mode where the machine operates semi-autonomously and the human overrides only when necessary. All interactions are instrumented and entered directly into a database using a standard interface and SQL. The collected data will typically be analyzed in the laboratory to quantify system performance, and to introduce algorithmic improvements prior to a new set of experiments. Over the course of system development iterations, the nature of the Wizard evolves over a continuum of functions from actual control (in the absence of any system knowledge) to passive observer. The user cannot tell the difference.
Cite as: Ammicht, E., Gorin, A., Alonso, T. (1999) Knowledge collection for natural language spoken dialog systems. Proc. 6th European Conference on Speech Communication and Technology (Eurospeech 1999), 1375-1378, doi: 10.21437/Eurospeech.1999-310
@inproceedings{ammicht99_eurospeech,
author={Egbert Ammicht and Allen Gorin and Tirso Alonso},
title={{Knowledge collection for natural language spoken dialog systems}},
year=1999,
booktitle={Proc. 6th European Conference on Speech Communication and Technology (Eurospeech 1999)},
pages={1375--1378},
doi={10.21437/Eurospeech.1999-310}
}