ISCA Archive ICSLP 2000
ISCA Archive ICSLP 2000

Discriminative training of tied-mixture HMM by deterministic annealing

Liang Gu, Jayanth Nayak, Kenneth Rose

A deterministic annealing algorithm for the design of tiedmixture HMM recognizers is proposed, which reduces the training sensitivity to parameter initialization, automatically smoothes the classification error cost function to allow gradientbased optimization, and seeks better solutions than known techniques. The new approach introduces randomness into the classification rule during the training process, and minimizes the expected error rate while controlling the level of randomness via a constraint on the Shannon entropy. As the entropy constraint is gradually relaxed, the effective cost function converges to the classification error rate and the system becomes a hard (nonrandom) recognizer. Experiments show that the proposed method outperforms design by maximum likelihood reestimation and by generalized probabilistic descent.


Cite as: Gu, L., Nayak, J., Rose, K. (2000) Discriminative training of tied-mixture HMM by deterministic annealing. Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), vol. 4, 183-186

@inproceedings{gu00e_icslp,
  author={Liang Gu and Jayanth Nayak and Kenneth Rose},
  title={{Discriminative training of tied-mixture HMM by deterministic annealing}},
  year=2000,
  booktitle={Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000)},
  pages={vol. 4, 183-186}
}