End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data. However, it is known that domain mismatch between training and testing data often leads to a degradation of recognition accuracy. In this work, we focus on the unsupervised domain adaptation for ASR and propose CMatch, a Character-level distribution matching method to perform fine-grained adaptation between each character in two domains. First, to obtain labels for the features belonging to each character, we achieve frame-level label assignment using the Connectionist Temporal Classification (CTC) pseudo labels. Then, we match the character-level distributions using Maximum Mean Discrepancy. We train our algorithm using the self-training technique. Experiments on the Libri-Adapt dataset show that our proposed approach achieves 14.39% and 16.50% relative Word Error Rate (WER) reduction on both cross-device and cross-environment ASR. We also comprehensively analyze the different strategies for frame-level label assignment and Transformer adaptations.
Cite as: Hou, W., Wang, J., Tan, X., Qin, T., Shinozaki, T. (2021) Cross-Domain Speech Recognition with Unsupervised Character-Level Distribution Matching. Proc. Interspeech 2021, 3425-3429, doi: 10.21437/Interspeech.2021-57
@inproceedings{hou21b_interspeech,
author={Wenxin Hou and Jindong Wang and Xu Tan and Tao Qin and Takahiro Shinozaki},
title={{Cross-Domain Speech Recognition with Unsupervised Character-Level Distribution Matching}},
year=2021,
booktitle={Proc. Interspeech 2021},
pages={3425--3429},
doi={10.21437/Interspeech.2021-57}
}