Auditory-Visual Speech Processing (AVSP) 2010
Hakone, Kanagawa, Japan
Our recent study  showed that culture modulates the manner of the multisensory integration of affective information. Specifically, Japanese are more tuned to vocal processing than Dutch in the multisensory perception of emotion. The current study aimed to extend the findings by adding an experiment and conducting further analyses on the results of their study. In the experiments, pairs of affective faces and voices, expressing either congruent or incongruent emotion, were presented simultaneously. In Experiment 1, although the performance on the facial judgment was very high, a selective interference that angry voice interfered with judgment of happy face was found for the in-group judgment of Japanese participants. Also, the interference effect of to-be-ignored face on the judgment of vocal expressions was smaller in the Japanese group than in the Dutch group. The results suggest that Japanese are more tuned to vocal processing in the multisensory perception of emotion, regardless of whether the difficulty is matched between judgments of facial and vocal expressions. In Experiment 2, the effect of to-be-ignored voice was larger for the in-group than out-group speakers when judging the facial expression, whereas the opposite result was found in the vocal judgment. The results suggest that the relative weight of facial and vocal expressions can be modulated by the familiarity with the speakers.
Index Terms: emotion perception, cultural difference
Bibliographic reference. Tanaka, Akihiro / Koizumi, Ai / Imai, Hisato / Hiramatsu, Saori / Hiramoto, Eriko / Gelder, Beatrice de (2010): "Cross-cultural differences in the multisensory perception of emotion", In AVSP-2010, paper S3-3.