Active Learning for Domain Classification in a Commercial Spoken Personal Assistant

Xi C. Chen, Adithya Sagar, Justine T. Kao, Tony Y. Li, Christopher Klein, Stephen Pulman, Ashish Garg, Jason D. Williams


We describe a method for selecting relevant new training data for the LSTM-based domain selection component of our personal assistant system. Adding more annotated training data for any ML system typically improves accuracy, but only if it provides examples not already adequately covered in the existing data. However, obtaining, selecting, and labeling relevant data is expensive. This work presents a simple technique that automatically identifies new helpful examples suitable for human annotation. Our experimental results show that the proposed method, compared with random-selection and entropy-based methods, leads to higher accuracy improvements given a fixed annotation budget. Although developed and tested in the setting of a commercial intelligent assistant, the technique is of wider applicability.


 DOI: 10.21437/Interspeech.2019-1315

Cite as: Chen, X.C., Sagar, A., Kao, J.T., Li, T.Y., Klein, C., Pulman, S., Garg, A., Williams, J.D. (2019) Active Learning for Domain Classification in a Commercial Spoken Personal Assistant. Proc. Interspeech 2019, 1478-1482, DOI: 10.21437/Interspeech.2019-1315.


@inproceedings{Chen2019,
  author={Xi C. Chen and Adithya Sagar and Justine T. Kao and Tony Y. Li and Christopher Klein and Stephen Pulman and Ashish Garg and Jason D. Williams},
  title={{Active Learning for Domain Classification in a Commercial Spoken Personal Assistant}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={1478--1482},
  doi={10.21437/Interspeech.2019-1315},
  url={http://dx.doi.org/10.21437/Interspeech.2019-1315}
}