FAAVSP - The 1st Joint Conference on Facial Analysis, Animation, and
Auditory-Visual Speech Processing

Vienna, Austria
September 11-13, 2015

Comparison of Multisensory Display Rules in Expressing Complex Emotions between Cultures

Sachiko Takagi (1), Shiho Miyazawa (2), Elisabeth Huis In 't Veld (3), Beatrice de Gelder (4), Akihiro Tanaka (1)

(1) Tokyo Woman’s Christian University, Japan (2) JEED, Japan
(3) Tilburg University, The Netherlands; (4) Maastricht University, The Netherlands

Previous studies have suggested that there are cultural differences in display rules and decoding rules of the emotions. In this study, we examined the cultural differences in the multisensory display rules of the six basic emotions (happiness, anger, disgust, sadness, fear and surprise) and the six complex emotions (interest, contempt, embarrassment, shame, guilt, and envy) between Japanese and Dutch. In the experiment, we used the six kinds of faces and voices showing the six basic emotions. Japanese and Dutch participants were asked to create audiovisual movies expressing each of twelve emotions by combining one of the six faces and one of the six voices. Our results showed cultural differences in expressing complex emotions. Specifically, there are two ways to express complex emotion by combining a face and voice showing the basic emotions: connection or substitution. Japanese participants tend to use the way of connection and combine the face and voice showing different emotions. On the other hand, Dutch participants tend to use the way of substitution and combine the face and voice showing the same basic emotions. Our findings indicate differential display rules and decoding rules between cultures in expressing complex emotions. Index Terms: facial expression, vocal expression, complex emotions, display rules

Full Paper

Bibliographic reference.  Takagi, Sachiko / Miyazawa, Shiho / Huis In 't Veld, Elisabeth / Gelder, Beatrice de / Tanaka, Akihiro (2015): "Comparison of multisensory display rules in expressing complex emotions between cultures", In FAAVSP-2015, 57-62.