Sixth European Conference on Speech Communication and Technology

Budapest, Hungary
September 5-9, 1999

The Deterministic Annealing Approach for Discriminative Continuous HMM Design

Cecile Gelin-Huet, Kenneth Rose, Ajit Rao

Signal Compression Lab, Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA, USA

We propose a deterministic annealing (DA) algorithm to design classifiers based on continuous observation hidden Markov models. The algorithm belongs to the class of minimum classification error (MCE) techniques that are known to outperform maximum likelihood (ML) design. Most MCE methods smooth the piecewise constant classification error cost to facilitate the use of local descent optimization methods, but are susceptible to the numerous shallow local minimum traps that riddle the cost surface. The DA approach employs randomization of the classification rule followed by minimization of the corresponding expected misclassification rate, while controlling the level of randomness via a constraint on the Shannon entropy. The effective cost function is smooth and converges to the MCE cost at the limit of zero entropy. The proposed algorithm significantly outperforms both standard ML and standard MCE design methods on the E-set database.

Keywords: Speech recognition, Discriminative training, Deterministic annealing, continuous HMM.

Full Paper (PDF)   Gnu-Zipped Postscript

Bibliographic reference.  Gelin-Huet, Cecile / Rose, Kenneth / Rao, Ajit (1999): "The deterministic annealing approach for discriminative continuous HMM design", In EUROSPEECH'99, 2717-2720.