7th International Conference on Spoken Language Processing

September 16-20, 2002
Denver, Colorado, USA

Automatic Sign Translation

Ying Zhang, Bing Zhao, Jie Yang, Alex Waibel

Carnegie Mellon University, USA

Large amounts of information is embedded in the natural scenes. Signs are good examples of objects in natural environments which have rich information content. In this paper, we present our efforts in the automatic sign translation. We describe the challenges in the automatic sign translation and introduce the architecture of our current system for automatic detection and translation of Chinese signs. Two data-driven machine translation methods: Example Based Machine Translation (EBMT) and Statistical Machine Translation (SMT) are compared for the task of translating Chinese signs into English. We report the experimental results of both methods that are trained from a small bilingual sign corpus combined with a bilingual glossary. The experiment results indicate that EBMT generates more correct translations while SMT is better at inferring unseen patterns. We are currently working on developing a multi-engine machine translation system that can incrementally learn from the data and combine the results from EBMT and SMT.


Full Paper

Bibliographic reference.  Zhang, Ying / Zhao, Bing / Yang, Jie / Waibel, Alex (2002): "Automatic sign translation", In ICSLP-2002, 645-648.