Prosodic cues for emotion: analysis with discrete characterization of intonation

Houwei Cao, Štefan Beňuš, Ruben C. Gur, Ragini Verma, Ani Nenkova


In this paper we study the relationship between acted perceptually unambiguous emotion and prosody. Unlike most contemporary approaches which base the analysis of emotion in voice solely on continuous features extracted automatically from the acoustic signal, we analyze the predictive power of discrete characterizations of intonations in the ToBI framework. The goal of our work is to test if particular discrete prosodic events provide significant discriminative power for emotion recognition. Our exper- iments provide strong evidence that patterns in breaks, boundary tones and type of pitch accent are highly informative of the emotional content of speech. We also present results from automatic prediction of emotion based on ToBI-derived features and compare their prediction power with state-of-the-art bag-of-frame acoustic features. Our results indicate their similar performance in the sentence-dependent emotion prediction tasks, while acoustic features are more robust for the sentence-independent tasks. Finally, we combine ToBI features and acoustic features together and further achieve modest improvements in sentence-independent emotion prediction, particularly in differentiating fear and neutral from other emotion.


 DOI: 10.21437/SpeechProsody.2014-14

Cite as: Cao, H., Beňuš, Š., Gur, R.C., Verma, R., Nenkova, A. (2014) Prosodic cues for emotion: analysis with discrete characterization of intonation. Proc. 7th International Conference on Speech Prosody 2014, 130-134, DOI: 10.21437/SpeechProsody.2014-14.


@inproceedings{Cao2014,
  author={Houwei Cao and Štefan Beňuš and Ruben C. Gur and Ragini Verma and Ani Nenkova},
  title={{Prosodic cues for emotion: analysis with discrete characterization of intonation}},
  year=2014,
  booktitle={Proc. 7th International Conference on Speech Prosody 2014},
  pages={130--134},
  doi={10.21437/SpeechProsody.2014-14},
  url={http://dx.doi.org/10.21437/SpeechProsody.2014-14}
}