International Symposium on Chinese Spoken Language Processing (ISCSLP 2002)

Taipei, Taiwan
August 23-24, 2002

A Study of Evaluation Method for Synthetic Mandarin Speech

Huaiqiao Bao (1), Anhong Wang (2), Shinan Lu (3)

(1) Institute of Nationality Studies, CASS, (2) Department of Chinese, Peking University, (3) Institute of Acoustics, Academia Sinica, Beijing, China

This study tries to put forward a kind of highly effective evaluation method according to the actuality of synthetic Mandarin. Through evaluating the two Mandarin TTS system of Beijing info-quick Sino-voice speech technique corp., the paper explores MOS (Mean Opinion score) method with a view of practicality. During operating, we defined the five-score rather accurately, and composed a set of testing materials. In addition, we trained the listeners on the basic knowledge of phonetics. The listeners participated in the judgment of speech of each score. Therefore, it increased the consistent understanding among different listeners, and avoided excessively disperse scores of different listeners. Moreover, during testing, the materials of two TTS system were set at random, and it can avoid learning effect for listeners. Besides MOS estimate, we also tried to use rather objective evaluating method. The result of assessment is reasonable.

Full Paper

Bibliographic reference.  Bao, Huaiqiao / Wang, Anhong / Lu, Shinan (2002): "A study of evaluation method for synthetic Mandarin speech", In ISCSLP 2002, paper 69.