5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Comparative Evaluation of Synthetic Prosody with the PURR Method

Gerit P. Sonntag, Thomas Portele

Institut fur Kommunikationsforschung und Phonetik (IKP), Universitat Bonn, Germany

In order to evaluate the prosodic output of a speech synthesis system independently from its segmental quality, we have developed a special way to delexicalize speech stimuli which we call PURR (Prosody Unveiling through Restricted Representation). We compared the use of PURR stimuli for the evaluation of prosodic naturalness in three different test designs: magnitude estimation (ME), categorical estimation (CE), and ranking order (RO). Sentences of different types were synthesized by six German synthesis systems. The synthetic utterances and one human voice were comparatively judged by experienced listeners. On the whole the results of all three methods are in good agreement. Choice of stimuli seems to be more important than the choice of method.

Full Paper

Bibliographic reference.  Sonntag, Gerit P. / Portele, Thomas (1998): "Comparative evaluation of synthetic prosody with the PURR method", In ICSLP-1998, paper 0018.