Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Recognition of Emotional States using Voice, Face Image and Thermal Image of Face

Tetsuro Kitazoe, Sung-Ill Kim, Yasunari Yoshitomi, Tatsuhiko Ikeda

Department of Computer Science and Systems Engineering, Faculty of Engineering, Miyazaki University, Japan

A new integration method is presented to recognize the emotional expressions. We attempted to use both voices and facial expressions. For voices, we use such prosodic parameters as pitch signals, energy, and their derivatives, which are trained by Hidden Markov Model (HMM) for recognition. For facial expressions, we use feature parameters from thermal images in addition to visible images, which are trained by neural networks (NN) for recognition. The thermal images are observed by infrared ray which is not influenced by lighting conditions. The total recognition rates show better performance than each performance rate obtained from isolated experiment. The results are compared with the recognition by human questionnaire.

Full Paper

Bibliographic reference.  Kitazoe, Tetsuro / Kim, Sung-Ill / Yoshitomi, Yasunari / Ikeda, Tatsuhiko (2000): "Recognition of emotional states using voice, face image and thermal image of face", In ICSLP-2000, vol.1, 653-656.