Neural Text-to-Speech with a Modeling-by-Generation Excitation Vocoder

Eunwoo Song, Min-Jae Hwang, Ryuichi Yamamoto, Jin-Seob Kim, Ohsung Kwon, Jae-Min Kim


This paper proposes a modeling-by-generation (MbG) excitation vocoder for a neural text-to-speech (TTS) system. Recently proposed neural excitation vocoders can realize qualified waveform generation by combining a vocal tract filter with a WaveNet-based glottal excitation generator. However, when these vocoders are used in a TTS system, the quality of synthesized speech is often degraded owing to a mismatch between training and synthesis steps. Specifically, the vocoder is separately trained from an acoustic model front-end. Therefore, estimation errors of the acoustic model are inevitably boosted throughout the synthesis process of the vocoder backend. To address this problem, we propose to incorporate an MbG structure into the vocoder’s training process. In the proposed method, the excitation signal is extracted by the acoustic model’s generated spectral parameters, and the neural vocoder is then optimized not only to learn the target excitation’s distribution but also to compensate for the estimation errors occurring from the acoustic model. Furthermore, as the generated spectral parameters are shared in the training and synthesis steps, their mismatch conditions can be reduced effectively. The experimental results verify that the proposed system provides high-quality synthetic speech by achieving a mean opinion score of 4.57 within the TTS framework.


 DOI: 10.21437/Interspeech.2020-2116

Cite as: Song, E., Hwang, M., Yamamoto, R., Kim, J., Kwon, O., Kim, J. (2020) Neural Text-to-Speech with a Modeling-by-Generation Excitation Vocoder. Proc. Interspeech 2020, 3570-3574, DOI: 10.21437/Interspeech.2020-2116.


@inproceedings{Song2020,
  author={Eunwoo Song and Min-Jae Hwang and Ryuichi Yamamoto and Jin-Seob Kim and Ohsung Kwon and Jae-Min Kim},
  title={{Neural Text-to-Speech with a Modeling-by-Generation Excitation Vocoder}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={3570--3574},
  doi={10.21437/Interspeech.2020-2116},
  url={http://dx.doi.org/10.21437/Interspeech.2020-2116}
}