ISCA Archive Interspeech 2021
ISCA Archive Interspeech 2021

Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning

Nilaksh Das, Sravan Bodapati, Monica Sunkara, Sundararajan Srinivasan, Duen Horng Chau

Training deep neural networks for automatic speech recognition (ASR) requires large amounts of transcribed speech. This becomes a bottleneck for training robust models for accented speech which typically contains high variability in pronunciation and other semantics, since obtaining large amounts of annotated accented data is both tedious and costly. Often, we only have access to large amounts of unannotated speech from different accents. In this work, we leverage this unannotated data to provide semantic regularization to an ASR model that has been trained only on one accent, to improve its performance for multiple accents. We propose Accent Pre-Training (Acc-PT), a semi-supervised training strategy that combines transfer learning and adversarial training. Our approach improves the performance of a state-of-the-art ASR model by 33% on average over the baseline across multiple accents, training only on annotated samples from one standard accent, and as little as 105 minutes of unannotated speech from a target accent.


doi: 10.21437/Interspeech.2021-1888

Cite as: Das, N., Bodapati, S., Sunkara, M., Srinivasan, S., Chau, D.H. (2021) Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning. Proc. Interspeech 2021, 1314-1318, doi: 10.21437/Interspeech.2021-1888

@inproceedings{das21b_interspeech,
  author={Nilaksh Das and Sravan Bodapati and Monica Sunkara and Sundararajan Srinivasan and Duen Horng Chau},
  title={{Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning}},
  year=2021,
  booktitle={Proc. Interspeech 2021},
  pages={1314--1318},
  doi={10.21437/Interspeech.2021-1888}
}