16th Annual Conference of the International Speech Communication Association

Dresden, Germany
September 6-10, 2015

Speaker Adaptation of Convolutional Neural Network Using Speaker Specific Subspace Vectors of SGMM

Murali Karthick B., Prateek Kolhar, S. Umesh

IIT Madras, India

The recent success of convolutional neural network (CNN) in speech recognition is due to its ability to capture translational variance in spectral features while performing discrimination. The CNN architecture requires correlated features as input and thus fMLLR transform which is estimated in de-correlated feature space fails to give significant improvement. In this paper, we propose two methods for extracting speaker adapted features in a correlated space using SGMMs. First, we estimate fMLLR transforms for correlated features by full covariance Gaussians using SGMM approach. Second, we augment speaker specific subspace vectors with acoustic features to provide speaker information in CNN models. Finally we propose a bottleneck - joint CNN/DNN framework to exploit the effects of both (fMLLR+ivectors) and (SGMM-fMLLR+speaker vectors) features. Experiments on TIMIT task show that our proposed features give 5.7% relative improvement over the log-mel features. Furthermore experiments on switchboard task show that the bottleneck - joint CNN/DNN model achieves 12.2% relative improvement over baseline joint CNN/DNN framework.

Full Paper

Bibliographic reference.  Karthick B., Murali / Kolhar, Prateek / Umesh, S. (2015): "Speaker adaptation of convolutional neural network using speaker specific subspace vectors of SGMM", In INTERSPEECH-2015, 1096-1100.