ISCA Archive SSW 2021
ISCA Archive SSW 2021

Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech

Raahil Shah, Kamil Pokora, Abdelhamid Ezzerg, Viacheslav Klimkov, Goeric Huybrechts, Bartosz Putrycz, Daniel Korzekwa, Thomas Merritt

Whilst recent neural text-to-speech (TTS) approaches produce high-quality speech, they typically require a large amount of recordings from the target speaker. In previous work [1], a 3- step method was proposed to generate high-quality TTS while greatly reducing the amount of data required for training. However, we have observed a ceiling effect in the level of naturalness achievable for highly expressive voices when using this approach. In this paper, we present a method for building highly expressive TTS voices with as little as 15 minutes of speech data from the target speaker. Compared to the current state-of-the-art approach, our proposed improvements close the gap to recordings by 23.3% for naturalness of speech and by 16.3% for speaker similarity. Further, we match the naturalness and speaker similarity of a Tacotron2-based full-data (≈ 10 hours) model using only 15 minutes of target speaker data, whereas with 30 minutes or more, we significantly outperform it. The following improvements are proposed: 1) changing from an autoregressive, attention-based TTS model to a nonautoregressive model replacing attention with an external duration model and 2) an additional Conditional Generative Adversarial Network (cGAN) based fine-tuning step.


doi: 10.21437/SSW.2021-17

Cite as: Shah, R., Pokora, K., Ezzerg, A., Klimkov, V., Huybrechts, G., Putrycz, B., Korzekwa, D., Merritt, T. (2021) Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech. Proc. 11th ISCA Speech Synthesis Workshop (SSW 11), 96-101, doi: 10.21437/SSW.2021-17

@inproceedings{shah21_ssw,
  author={Raahil Shah and Kamil Pokora and Abdelhamid Ezzerg and Viacheslav Klimkov and Goeric Huybrechts and Bartosz Putrycz and Daniel Korzekwa and Thomas Merritt},
  title={{Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech}},
  year=2021,
  booktitle={Proc. 11th ISCA Speech Synthesis Workshop (SSW 11)},
  pages={96--101},
  doi={10.21437/SSW.2021-17}
}