Third Workshop on Spoken Language Technologies for Under-resourced Languages

Cape Town, South Africa
May 7-9, 2012

Integration of Language Identification into a Recognition System for Spoken Conversations Containing Code-Switches

Jochen Weiner (1), Ngoc Thang Vu (1), Dominic Telaar (1), Florian Metze (3), Tanja Schultz (1,3), Dau-Cheng Lyu (2), Eng-Siong Chng (2), Haizhou Li (2)

(1) Cognitive Systems Lab, Institute for Anthropomatics, Karlsruhe Institute of Technology (KIT)
(2) School of Computer Engineering, Nanyang Technological University (NTU), Singapore
(3) Language Technologies Institute, Carnegie Mellon University (CMU), Pittsburgh, PA, USA

This paper describes the integration of language identification (LID) into a multilingual automatic speech recognition (ASR) system for spoken conversations containing code-switches between Mandarin and English. We apply a multistream approach to combine at frame level the acoustic model score and the language information, where the latter is provided by an LID component. Furthermore, we advance this multistream approach by a new method called “Language Lookahead”, in which the language information of subsequent frames is used to improve accuracy. Both methods are evaluated using a set of controlled LID results with varying frame accuracies. Our results show that both approaches improve the ASR performance by at least 4% relative if the LID achieves a minimum frame accuracy of 85%.

Index Terms: code-switching, multi-stream combination, language lookahead

Full Paper

Bibliographic reference.  Weiner, Jochen / Vu, Ngoc Thang / Telaar, Dominic / Metze, Florian / Schultz, Tanja / Lyu, Dau-Cheng / Chng, Eng-Siong / Li, Haizhou (2012): "Integration of language identification into a recognition system for spoken conversations containing code-Switches", In SLTU-2012, 76-79.