14thAnnual Conference of the International Speech Communication Association

Lyon, France
August 25-29, 2013

Deep Belief Network Based Semantic Taggers for Spoken Language Understanding

Anoop Deoras, Ruhi Sarikaya

Microsoft Corporation, USA

This paper investigates the use of deep belief networks (DBN) for semantic tagging, a sequence classification task, in spoken language understanding (SLU).We evaluate the performance of the DBN based sequence tagger on the well-studied ATIS task and compare our technique to conditional random fields (CRF), a stateof- the-art classifier for sequence classification. In conjunction with lexical and named entity features, we also use dependency parser based syntactic features and part of speech (POS) tags. Under both noisy conditions (output of automatic speech recognition system) and clean conditions (manual transcriptions), our deep belief network based sequence tagger outperforms the best CRF based system described in [1] by an absolute 2% and 1% F-measure, respectively. Upon carrying out an analysis of cases where CRF and DBN models made different predictions, we observed that when discrete features are projected onto a continuous space during neural network training, the model learns to cluster these features leading to its improved generalization capability, relative to a CRF model, especially in cases where some features are either missing or noisy.


  1. G. Tur, D. Hakkani-Tur, L. Heck, and S. Parthasarathy, "Sentence Simplification for Spoken Language Understanding," in Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2011, 56285631.

Full Paper

Bibliographic reference.  Deoras, Anoop / Sarikaya, Ruhi (2013): "Deep belief network based semantic taggers for spoken language understanding", In INTERSPEECH-2013, 2713-2717.