ISCA Archive AVSP 2013
ISCA Archive AVSP 2013

Transforming neutral visual speech into expressive visual speech

Felix Shaw, Barry-John Theobald

We present a method for transforming neutral visual speech sequences into realistic expressive visual speech sequences. By applying Independent Component Analysis (ICA) to visual features extracted from time aligned neutral and equivalent expressive sequences, a model that separates speech from expression can be learned. Analyzing the behavior of different speaking styles in terms of this model provides both a means for identifying the component(s) responsible for expression, and for learning the correspondence between different speaking styles. Exploiting this correspondence to transform neutral visual speech into expressive visual speech creates sequences that have the same time varying expressive dynamics as the equivalent ground-truth sequences, and an objective analysis shows that the neutral ICA parameters are shifted into the appropriate ranges for expressive visual speech.

Index Terms: expressive visual speech synthesis, independent component analysis, expressive style transformation


Cite as: Shaw, F., Theobald, B.-J. (2013) Transforming neutral visual speech into expressive visual speech. Proc. Auditory-Visual Speech Processing, 203-208

@inproceedings{shaw13_avsp,
  author={Felix Shaw and Barry-John Theobald},
  title={{Transforming neutral visual speech into expressive visual speech}},
  year=2013,
  booktitle={Proc. Auditory-Visual Speech Processing},
  pages={203--208}
}