11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Study on Interaction Between Entropy Pruning and Kneser-Ney Smoothing

Ciprian Chelba, Thorsten Brants, Will Neveitt, Peng Xu

Google, USA

The paper presents an in-depth analysis of a less known interaction between Kneser-Ney smoothing and entropy pruning that leads to severe degradation in language model performance under aggressive pruning regimes. Experiments in a data-rich setup such as voice search show a significant impact in WER as well: pruning Kneser-Ney and Katz models to 0.1% of their original impacts speech recognition accuracy significantly, approx. 10% relative.

Full Paper

Bibliographic reference.  Chelba, Ciprian / Brants, Thorsten / Neveitt, Will / Xu, Peng (2010): "Study on interaction between entropy pruning and kneser-ney smoothing", In INTERSPEECH-2010, 2422-2425.