Based on the performance assessment of speech synthesis systems for Chinese the total quality evaluation of them has been carried out regular since 1994. The total quality evaluation includes speech intelligibility test at different levels (syllable, word and sentence), speech naturalness test and anti-interference ability test for phonetic module and text processing ability test for linguistic module. The designing principle of testing materials and testing methods are briefly described and the test results of four text-to-speech (TTS) systems for Chinese are presented in this paper. It is shown that 1. All professional technicians joining the speech naturalness test with the testing crew together overestimated the overall quality of the four tested systems; 2. The word intelligibility and Semantically Unpredicted Sentence (SUS) score are good for evaluating speech synthesis systems; 3. The anti-interference ability of synthetic speech is rather weak about 20 per cent in syllable intelligibility lower than natural speech under condition of S/N=5dB.
Cite as: Zhang, J., Dong, S., Yu, G. (1998) Total quality evaluation of speech synthesis systems. Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998), paper 0060, doi: 10.21437/ICSLP.1998-12
@inproceedings{zhang98_icslp,
author={Jialu Zhang and Shiwei Dong and Ge Yu},
title={{Total quality evaluation of speech synthesis systems}},
year=1998,
booktitle={Proc. 5th International Conference on Spoken Language Processing (ICSLP 1998)},
pages={paper 0060},
doi={10.21437/ICSLP.1998-12}
}