15th Annual Conference of the International Speech Communication Association

September 14-18, 2014

Hypotheses Ranking for Robust Domain Classification and Tracking in Dialogue Systems

Jean-Philippe Robichaud, Paul A. Crook, Puyang Xu, Omar Zia Khan, Ruhi Sarikaya

Microsoft, USA

We present a novel application of hypothesis ranking (HR) for the task of domain detection in a multi-domain, multi-turn dialog system. Alternate, domain dependent, semantic frames from a spoken language understanding (SLU) analysis are ranked using a gradient boosted decision trees (GBDT) ranker to determine the most likely domain. The ranker, trained using Lambda Rank, makes use of a range of signals derived from the SLU and previous turn context to improve domain detection. On a multi-turn corpus we show that this approach offers accuracy improvements of 3.2% absolute (25.6% relative) compared to relying solely on upfront non-contextual SLU domain models and 2.9% (24.5% relative) improvement even with contextual SLU domain models. We also show that HR can be trained to be robust to changes in the SLU.

Full Paper

Bibliographic reference.  Robichaud, Jean-Philippe / Crook, Paul A. / Xu, Puyang / Khan, Omar Zia / Sarikaya, Ruhi (2014): "Hypotheses ranking for robust domain classification and tracking in dialogue systems", In INTERSPEECH-2014, 145-149.