10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

HMM-Based Automatic Eye-Blink Synthesis from Speech

Michal Dziemianko, Gregor Hofer, Hiroshi Shimodaira

University of Edinburgh, UK

In this paper we present a novel technique to automatically synthesise eye blinking from a speech signal. Animating the eyes of a talking head is important as they are a major focus of attention during interaction. The developed system predicts eye blinks from the speech signal and generates animation trajectories automatically employing a “Trajectory Hidden Markov Model”. The evaluation of the recognition performance showed that the timing of blinking can be predicted from speech with an F-score value upwards of 52%, which is well above chance. Additionally, a preliminary perceptual evaluation was conducted, that confirmed that adding eye blinking significantly improves the perception the character. Finally it showed that the speech synchronised synthesised blinks outperform random blinking in naturalness ratings.

Full Paper

Bibliographic reference.  Dziemianko, Michal / Hofer, Gregor / Shimodaira, Hiroshi (2009): "HMM-based automatic eye-blink synthesis from speech", In INTERSPEECH-2009, 1799-1802.