15th Annual Conference of the International Speech Communication Association

September 14-18, 2014

Towards Speaker Adaptive Training of Deep Neural Network Acoustic Models

Yajie Miao, Hao Zhang, Florian Metze

Carnegie Mellon University, USA

We investigate the concept of speaker adaptive training (SAT) in the context of deep neural network (DNN) acoustic models. Previous studies have shown success of performing speaker adaptation for DNNs in speech recognition. In this paper, we apply SAT to DNNs by learning two types of feature mapping neural networks. Given an initial DNN model, these networks take speaker i-vectors as additional information and project DNN inputs into a speaker-normalized space. The final SAT model is obtained by updating the canonical DNN in the normalized feature space. Experiments on a Switchboard 110-hour setup show that compared with the baseline DNN, the SAT-DNN model brings 7.5% and 6.0% relative improvement when DNN inputs are speaker-independent and speaker-adapted features respectively. Further evaluations on the more challenging BABEL datasets reveal significant word error rate reduction achieved by SAT-DNN.

Full Paper

Bibliographic reference.  Miao, Yajie / Zhang, Hao / Metze, Florian (2014): "Towards speaker adaptive training of deep neural network acoustic models", In INTERSPEECH-2014, 2189-2193.