11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

A Regularized Discriminative Training Method of Acoustic Models Derived by Minimum Relative Entropy Discrimination

Yotaro Kubo (1), Shinji Watanabe (2), Atsushi Nakamura (2), Tetsunori Kobayashi (1)

(1) Waseda University, Japan
(2) NTT Corporation, Japan

We present a realization method of the principle of minimum relative entropy discrimination (MRED) in order to derive a regularized discriminative training method. MRED is advantageous since it provides a Bayesian interpretations of the conventional discriminative training methods and regularization techniques. In order to realize MRED for speech recognition, we proposed an approximation method of MRED that strictly preserves the constraints used in MRED. Further, in order to practically perform MRED, an optimization method based on convex optimization and its solver based on the cutting plane algorithm are also proposed. The proposed methods were evaluated on continuous phoneme recognition tasks. We confirmed that the MRED-based training system outperformed conventional discriminative training methods in the experiments.

Full Paper

Bibliographic reference.  Kubo, Yotaro / Watanabe, Shinji / Nakamura, Atsushi / Kobayashi, Tetsunori (2010): "A regularized discriminative training method of acoustic models derived by minimum relative entropy discrimination", In INTERSPEECH-2010, 2954-2957.