8th European Conference on Speech Communication and Technology

Geneva, Switzerland
September 1-4, 2003


Multimodal Interaction on PDA's Integrating Speech and Pen Inputs

Sorin Dusan (1), Gregory J. Gadbois (2), James Flanagan (1)

(1) Rutgers University, USA
(2) HandHeld Speech LLC, USA

Recent efforts in the field of mobile computing are directed toward speech-enabling portable computers. This paper presents a method of multimodal interaction and an application which integrates speech and pen on mobile computers. The application is designed for documenting traffic accident diagrams by police. The novelty of this application is due to a) its method of fusing the speech and pen inputs, and b) its fully embedded speech engine. Preliminary experiments showed flexibility, versatility and increased naturalness and user satisfaction during multimodal interaction.

Full Paper

Bibliographic reference.  Dusan, Sorin / Gadbois, Gregory J. / Flanagan, James (2003): "Multimodal interaction on PDA's integrating speech and pen inputs", In EUROSPEECH-2003, 2225-2228.