9th Annual Conference of the International Speech Communication Association

Brisbane, Australia
September 22-26, 2008

An Ellipsoid Constrained Quadratic Programming Perspective to Discriminative Training of HMMs

Peng Liu, Frank K. Soong

Microsoft Research Asia, China

In this paper, we reformulate the optimization in discriminative training (DT) of HMMs as an ellipsoid constrained quadratic programming (ECQP) problem, where a second order of the non-linear space is adopted. We show that the unique optimal solution of ECQP can be obtained by an efficient line search and no relaxation is needed as in a general quadratically constrained quadratic programming (QCQP). Moreover, a subspace combination condition is introduced to further simplify it under certain cases. The concrete ECQP form of DT of HMMs is given based on a locality constraint and reasonable assumptions, and the algorithm can be conducted to update Gaussians jointly or separately in either sequential or batch mode. Under the perspective of ECQP, relationships between various popular DT optimization algorithms are discussed. Experimental results on two recognition tasks show that ECQP considerably outperforms other popular algorithms in terms of final recognition accuracy and convergence speed in iterations.

Full Paper

Bibliographic reference.  Liu, Peng / Soong, Frank K. (2008): "An ellipsoid constrained quadratic programming perspective to discriminative training of HMMs", In INTERSPEECH-2008, 281-284.