COST278 and ISCA Tutorial and Research Workshop (ITRW) on Robustness Issues in Conversational Interaction

University of East Anglia, Norwich, UK
August 30-31, 2004

Robust Multimodal Interaction in Challenging Environments

Philip R. Cohen, Sanjeev Kumar, Ed Kaiser

Center for Human-Computer Communication, Oregon Health and Science University, USA

This paper briefly summarizes recent research documenting increased robustness that results from using multimodal interaction in environments that would challenge current graphical user interfaces, or even interfaces based on single natural communication modalities, such as speech or gesture. Multimodal interfaces that support the mutual disambiguation of modalities [1,2,3] can use one or more modalities or sources of information to overcome errors in another. The results presented in this paper demonstrate this phenomenon for circumstances in which users are highly exerted, and for 3D augmented and virtual reality environments that offer 4 sources of information.

Full Paper

Bibliographic reference.  Cohen, Philip R. / Kumar, Sanjeev / Kaiser, Ed (2004): "Robust multimodal interaction in challenging environments", In Robust2004, paper KPC.