INTERSPEECH 2009
10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

A Fast Online Algorithm for Large Margin Training of Continuous Density Hidden Markov Models

Chih-Chieh Cheng (1), Fei Sha (2), Lawrence K. Saul (1)

(1) University of California at San Diego, USA
(2) University of Southern California, USA

We propose an online learning algorithm for large margin training of continuous density hidden Markov models. The online algorithm updates the model parameters incrementally after the decoding of each training utterance. For large margin training, the algorithm attempts to separate the log-likelihoods of correct and incorrect transcriptions by an amount proportional to their Hamming distance. We evaluate this approach to hidden Markov modeling on the TIMIT speech database. We find that the algorithm yields significantly lower phone error rates than other approachesóboth online and batch ó that do not attempt to enforce a large margin. We also find that the algorithm converges much more quickly than analogous batch optimizations for large margin training.

Full Paper

Bibliographic reference.  Cheng, Chih-Chieh / Sha, Fei / Saul, Lawrence K. (2009): "A fast online algorithm for large margin training of continuous density hidden Markov models", In INTERSPEECH-2009, 668-671.