Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Discriminative Training of Tied-Mixture HMM by Deterministic Annealing

Liang Gu, Jayanth Nayak, Kenneth Rose

Department of Electrical and Computer Engineering University of California, Santa Barbara, CA, USA

A deterministic annealing algorithm for the design of tiedmixture HMM recognizers is proposed, which reduces the training sensitivity to parameter initialization, automatically smoothes the classification error cost function to allow gradientbased optimization, and seeks better solutions than known techniques. The new approach introduces randomness into the classification rule during the training process, and minimizes the expected error rate while controlling the level of randomness via a constraint on the Shannon entropy. As the entropy constraint is gradually relaxed, the effective cost function converges to the classification error rate and the system becomes a hard (nonrandom) recognizer. Experiments show that the proposed method outperforms design by maximum likelihood reestimation and by generalized probabilistic descent.

Full Paper

Bibliographic reference.  Gu, Liang / Nayak, Jayanth / Rose, Kenneth (2000): "Discriminative training of tied-mixture HMM by deterministic annealing", In ICSLP-2000, vol.4, 183-186.