14thAnnual Conference of the International Speech Communication Association

Lyon, France
August 25-29, 2013

A Preliminary Study of Cross-Lingual Emotion Recognition from Speech: Automatic Classification versus Human Perception

Je Hun Jeon, Duc Le, Rui Xia, Yang Liu

University of Texas at Dallas, USA

The aim of this study is to investigate the effect of cross-lingual data on human perception and automatic classification of emotion from speech. We use four different databases from three languages (English, Chinese, and German) and two types (acted and improvised). For automatic classification, there is a significant degradation using cross-corpus than within-corpus setup. For human perception, we observe differences between native and non-native speakers when judging emotions for a language, and there is less performance loss in cross-language setup compared to automatic classification. In addition, we find that the automatic approaches work well in classifying the emotional activation category: positive and negative activated emotions, but are not good at classifying instances within the same activation category, which is different from the confusion patterns of the human perception experiment. This study provides insights to better understanding of cross-lingual human emotion perception and development of robust automatic emotion recognition systems.

Full Paper

Bibliographic reference.  Jeon, Je Hun / Le, Duc / Xia, Rui / Liu, Yang (2013): "A preliminary study of cross-lingual emotion recognition from speech: automatic classification versus human perception", In INTERSPEECH-2013, 2837-2840.