Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissions

Rohan Kumar Das, Tomi Kinnunen, Wen-Chin Huang, Zhen-Hua Ling, Junichi Yamagishi, Zhao Yi, Xiaohai Tian, Tomoki Toda


The Voice Conversion Challenge 2020 is the third edition under its flagship that promotes intra-lingual semiparallel and crosslingual voice conversion (VC). While the primary evaluation of the challenge submissions was done through crowd-sourced listening tests, we also performed an objective assessment of the submitted systems. The aim of the objective assessment is to provide complementary performance analysis that may be more beneficial than the time-consuming listening tests. In this study, we examined five types of objective assessments using automatic speaker verification (ASV), neural speaker embeddings, spoofing countermeasures, predicted mean opinion scores (MOS), and automatic speech recognition (ASR). Each of these objective measures assesses the VC output along different aspects. We observed that the correlations of these objective assessments with the subjective results were high for ASV, neural speaker embedding, and ASR, which makes them more influential for predicting subjective test results. In addition, we performed spoofing assessments on the submitted systems and identified some of the VC methods showing a potentially high security risk.


 DOI: 10.21437/VCC_BC.2020-15

Cite as: Das, R.K., Kinnunen, T., Huang, W., Ling, Z., Yamagishi, J., Yi, Z., Tian, X., Toda, T. (2020) Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissions. Proc. Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020, 99-120, DOI: 10.21437/VCC_BC.2020-15.


@inproceedings{Das2020,
  author={Rohan Kumar Das and Tomi Kinnunen and Wen-Chin Huang and Zhen-Hua Ling and Junichi Yamagishi and Zhao Yi and Xiaohai Tian and Tomoki Toda},
  title={{Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissions}},
  year=2020,
  booktitle={Proc. Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020},
  pages={99--120},
  doi={10.21437/VCC_BC.2020-15},
  url={http://dx.doi.org/10.21437/VCC_BC.2020-15}
}