12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

Automatically Optimizing Utterance Classification Performance without Human in the Loop

Yun-Cheng Ju, Jasha Droppo

Microsoft Research, USA

The Utterance Classification (UC) method has become a developer's choice over traditional Context Free Grammars (CFGs) for voice menus in telephony applications. This data driven method achieves higher accuracy and has great potential to utilize a huge amount of labeled training data. But, having a human manually label the training data can be expensive. This paper provides a robust recipe for training a UC system using inexpensive acoustic data with limited transcriptions or semantic labels. It also describes two new algorithms that use caller confirmation, which naturally occurred within a dialog, to generate pseudo semantic labels. Experimental results show that, after having sufficient labeled data to achieve a reasonable accuracy, both of our algorithms can use unlabeled data to achieve the same performance as a system trained with labeled data, while completely eliminating the need for human supervision.

Full Paper

Bibliographic reference.  Ju, Yun-Cheng / Droppo, Jasha (2011): "Automatically optimizing utterance classification performance without human in the loop", In INTERSPEECH-2011, 721-724.