Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

Objective Methods for Evaluating Synthetic Intonation

Robert A.J. Clark, Kurt E. Dusterhoff

Centre for Speech Technology Research, University of Edinburgh, Edinburgh, UK

This paper describes the development and evaluation of objective methods for testing synthetic intonation. While subjective methods are available for assessing the quality of synthetic intonation, such tests consume time and resources, and are not useful for day-to-day model development. Therefore, objective measures of F0 modelling are necessary. Currently, objective evaluation of synthetic intonation involves the use of Root Mean Squared Error and Correlation. However, it is unclear how large an improvement in either score must be before it is reflected perceptually. It is also unclear how detailed an analysis these metrics provide. Therefore, two other metrics are to be tested, both of which are similar to a basic RMSE measurement. All of the evaluation results are compared to a perceptual study in order to determine how the objective measures relate to perceived differences in the contours.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Clark, Robert A.J. / Dusterhoff, Kurt E. (1999): "Objective methods for evaluating synthetic intonation", In EUROSPEECH'99, 1623-1626.