16th Annual Conference of the International Speech Communication Association

Dresden, Germany
September 6-10, 2015

A Study of Speaker Adaptation for DNN-Based Speech Synthesis

Zhizheng Wu, Pawel Swietojanski, Christophe Veaux, Steve Renals, Simon King

University of Edinburgh, UK

A major advantage of statistical parametric speech synthesis (SPSS) over unit-selection speech synthesis is its adaptability and controllability in changing speaker characteristics and speaking style. Recently, several studies using deep neural networks (DNNs) as acoustic models for SPSS have shown promising results. However, the adaptability of DNNs in SPSS has not been systematically studied. In this paper, we conduct an experimental analysis of speaker adaptation for DNN-based speech synthesis at different levels. In particular, we augment a low-dimensional speaker-specific vector with linguistic features as input to represent speaker identity, perform model adaptation to scale the hidden activation weights, and perform a feature space transformation at the output layer to modify generated acoustic features. We systematically analyse the performance of each individual adaptation technique and that of their combinations. Experimental results confirm the adaptability of the DNN, and listening tests demonstrate that the DNN can achieve significantly better adaptation performance than the hidden Markov model (HMM) baseline in terms of naturalness and speaker similarity.

Full Paper

Bibliographic reference.  Wu, Zhizheng / Swietojanski, Pawel / Veaux, Christophe / Renals, Steve / King, Simon (2015): "A study of speaker adaptation for DNN-based speech synthesis", In INTERSPEECH-2015, 879-883.