General–purpose state-of-the-art automatic speech recognition (ASR) systems have notably improved their quality in the last decade opening the possibility to be used in different practical applications, such as pronunciation assessment. However, the assessment of short words as minimal pairs in segmental approaches remains an important challenge for ASR, even more for non-native speakers. In this work, we use both our own tailored specific–purpose Kaldi–based ASR system and Google ASR to assess Spanish minimal pair words produced by 33 native Japanese speakers and to discuss their performance for computer-assisted pronunciation training (CAPT). Participants were split into three groups: experimental, in-classroom, and placebo. First two groups followed a pre/post-test training protocol spanning four weeks. Both the experimental and in-classroom groups achieved statistically significant differences at the end of the experiment, assessed by both ASR systems. We also found moderate correlation values between Google and Kaldi ASR systems in the pre/post-test values, and strong correlations between the post-test scores of both ASR systems and the CAPT application scores at the end of the experiment. Tailored ASR systems can bring clear benefits for a detailed study of pronunciation errors and results showed that they can be as useful as general–purpose ASR for assessing minimal pairs in CAPT tools.
Cite as: Tejedor-García, C., Cardeñoso-Payo, V., Escudero-Mancebo, D. (2021) Performance Comparison of Specific and General-Purpose ASR Systems for Pronunciation Assessment of Japanese Learners of Spanish. Proc. IberSPEECH 2021, 6-10, doi: 10.21437/IberSPEECH.2021-2
@inproceedings{tejedorgarcia21b_iberspeech,
author={Cristian Tejedor-García and Valentín Cardeñoso-Payo and David Escudero-Mancebo},
title={{Performance Comparison of Specific and General-Purpose ASR Systems for Pronunciation Assessment of Japanese Learners of Spanish}},
year=2021,
booktitle={Proc. IberSPEECH 2021},
pages={6--10},
doi={10.21437/IberSPEECH.2021-2}
}