Challenges for Robust Prosody-based Affect Recognition

Heather Pon-Barry, Arun Reddy Nelakurthi


Prosody-based affect recognition has great potential impact for building adaptive speech interfaces. For example, in intelligent systems for personalized learning, sensing a student’s level of certainty, which is often signaled prosodically, is one of the most interesting states to interpret and respond to. However, robust uncertainty recognition faces several challenges, including the lack of gold-standard labels, and differences in expressivity among speakers. In this paper we explore the intersection of these two issues. We have collected a corpus of spontaneous speech in a question-answering task. Three kinds of certainty labels are associated with each utterance. First, speakers rated their own level of certainty. Second, a panel of listeners rated how certain the speaker sounded. Third, an externally crowdsourced difficulty score is generated for each stimulus (the question). We present an analysis of the prosodic characteristics of individual speaking styles, as they relate to these three different measurements of certainty.


 DOI: 10.21437/SpeechProsody.2014-17

Cite as: Pon-Barry, H., Nelakurthi, A.R. (2014) Challenges for Robust Prosody-based Affect Recognition. Proc. 7th International Conference on Speech Prosody 2014, 144-148, DOI: 10.21437/SpeechProsody.2014-17.


@inproceedings{Pon-Barry2014,
  author={Heather Pon-Barry and Arun Reddy Nelakurthi},
  title={{Challenges for Robust Prosody-based Affect Recognition}},
  year=2014,
  booktitle={Proc. 7th International Conference on Speech Prosody 2014},
  pages={144--148},
  doi={10.21437/SpeechProsody.2014-17},
  url={http://dx.doi.org/10.21437/SpeechProsody.2014-17}
}