Recent studies in our lab show that emotions in speech are manifested as, besides supra-segmental trends, distinct variations in phoneme-level prosodic and spectral parameters. In this paper, we further investigate the significance of this finding in the context of emotional speech synthesis. Specifically, we study phonemelevel signal property manipulation in transforming the emotional information conveyed in a speech utterance. We analyze the effect of individual and combined modifications of F0, duration, energy and spectrum using data recorded by a professional actress with happy, angry, sad and neutral expressiveness. We use content matched source-target pairs and apply TDPSOLA for prosody and LPC for spectrum modifications by directly extracting the required parameters from the target speech. Listening tests conducted with 10 naive raters show that modi- fication of prosody and spectral envelope parameters by themselves is not sufficient. However, when applied together, modifying spectrum and prosody at the phone level gives successful results for most emotion pairs, except conversion to happy targets. We also observe that at the phoneme level, spectral envelope modifications are more effective than local prosodic modifications; and that, duration modifications are more effective than pitch modifications. The results confirm our hypothesis that phoneme level modifications can be used to fine tune the ensuing suprasegmental-parameter-based modifications to improve the overall quality of synthesized emotions.
Cite as: Bulut, M., Busso, C., Yildirim, S., Kazemzadeh, A., Lee, C.M., Lee, S., Narayanan, S. (2005) Investigating the role of phoneme-level modifications in emotional speech resynthesis. Proc. Interspeech 2005, 801-804, doi: 10.21437/Interspeech.2005-378
@inproceedings{bulut05_interspeech,
author={Murtaza Bulut and Carlos Busso and Serdar Yildirim and Abe Kazemzadeh and Chul Min Lee and Sungbok Lee and Shrikanth Narayanan},
title={{Investigating the role of phoneme-level modifications in emotional speech resynthesis}},
year=2005,
booktitle={Proc. Interspeech 2005},
pages={801--804},
doi={10.21437/Interspeech.2005-378}
}