8th Annual Conference of the International Speech Communication Association

Antwerp, Belgium
August 27-31, 2007

A Semi-Supervised Learning Approach for Morpheme Segmentation for an Arabic Dialect

Mei Yang (1), Jing Zheng (2), Andreas Kathol (2)

(1) University of Washington, USA
(2) SRI International, USA

We present a semi-supervised learning approach which utilizes a heuristic model for learning morpheme segmentation for Arabic dialects. We evaluate our approach by applying morpheme segmentation to the training data of a statistical machine translation (SMT) system. Experiments show that our approach is less sensitive to the availability of annotated stems than a previous rule-based approach and learns 12% more segmentations on our Iraqi Arabic data. When applied in an SMT system, our approach yields a 8% relative reduction in the training vocabulary size and a 0.8% relative reduction in the out-of-vocabulary (OOV) rate on the test set, again as compared to the rule-based approach. Finally, our approach also results in a modest increase in BLEU scores.

Full Paper

Bibliographic reference.  Yang, Mei / Zheng, Jing / Kathol, Andreas (2007): "A semi-supervised learning approach for morpheme segmentation for an Arabic dialect", In INTERSPEECH-2007, 1501-1504.