Investigation of Semi-Supervised Acoustic Model Training Based on the Committee of Heterogeneous Neural Networks

Naoyuki Kanda, Shoji Harada, Xugang Lu, Hisashi Kawai


This paper investigates the semi-supervised training for deep neural network-based acoustic models (AM). In the conventional self-learning approach, a “seed-AM” is first trained by using a small transcribed data set. Then, a large untranscribed data set is decoded by using the seed-AM to create a transcription, which is finally used to train a new AM on the entire data. Our investigation in this paper focuses on the different approach that uses additional complementary AMs to form a committee of label creation for untranscribed data. Especially, we investigate the case of using heterogeneous neural networks as complementary AMs, and the case of intentional exclusion of the primary seed-AM from the committee, both of which could enhance the chance to find more informative training samples for the seed-AM. We investigated those approaches based on Japanese lecture recognition experiments with 50-hours of transcribed data and 190-hours of untranscribed data. In our experiment, the committee-based approach showed significant improvements in the word error rate, and the best method finally recovered 75.2% of the oracle improvement with full manual transcription, while the conventional self-learning approach recovered only 32.7% of the oracle gain.


DOI: 10.21437/Interspeech.2016-72

Cite as

Kanda, N., Harada, S., Lu, X., Kawai, H. (2016) Investigation of Semi-Supervised Acoustic Model Training Based on the Committee of Heterogeneous Neural Networks. Proc. Interspeech 2016, 1325-1329.

Bibtex
@inproceedings{Kanda+2016,
author={Naoyuki Kanda and Shoji Harada and Xugang Lu and Hisashi Kawai},
title={Investigation of Semi-Supervised Acoustic Model Training Based on the Committee of Heterogeneous Neural Networks},
year=2016,
booktitle={Interspeech 2016},
doi={10.21437/Interspeech.2016-72},
url={http://dx.doi.org/10.21437/Interspeech.2016-72},
pages={1325--1329}
}