ISCA Archive Interspeech 2008
ISCA Archive Interspeech 2008

A probabilistic trajectory synthesis system for synthesising visual speech

Barry-John Theobald, Nicholas Wilkinson

We describe an unsupervised probabilistic approach for synthesising visual speech from audio. Acoustic features representing a training corpus are clustered and the probability density function (PDF) of each cluster is modelled as a Gaussian mixture model (GMM). A visual target in the form of a short-term parameter trajectory is generated for each cluster. Synthesis involves combining the cluster targets based on the likelihood of novel acoustic feature vectors, then cross-blending neighbouring regions of the synthesised short-term trajectories. The advantage of our approach is coarticulation effects are explicitly captured by the mapping. The influence of cluster targets naturally increase and decrease with the likelihood of the acoustic feature vectors.


doi: 10.21437/Interspeech.2008-183

Cite as: Theobald, B.-J., Wilkinson, N. (2008) A probabilistic trajectory synthesis system for synthesising visual speech. Proc. Interspeech 2008, 1857-1860, doi: 10.21437/Interspeech.2008-183

@inproceedings{theobald08_interspeech,
  author={Barry-John Theobald and Nicholas Wilkinson},
  title={{A probabilistic trajectory synthesis system for synthesising visual speech}},
  year=2008,
  booktitle={Proc. Interspeech 2008},
  pages={1857--1860},
  doi={10.21437/Interspeech.2008-183}
}