5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Enhancing a WIMP Based Interface With Speech, Gaze Tracking and Agents

Lau Bakman, Mads Blidegn, Martin Wittrup, Lars Bo Larsen, Thomas B. Moeslund

Aalborg University, Denmark

This paper describes an attempt to enhance a windows based (WIMP - Windows Icon Menu Pointer) environment. The goal is to establish whether user interaction on the common desktop PC can be augmented by adding new modalities to the WIMP interface, thus bridging the gap between todays interaction patterns and future interfaces comprising e.g. advanced conversational capabilities, VR technology, etc. A user survey was carried out to establish the trouble spots of the WIMP interface on the most common desktop work station, the Windows 95 PC. On the basis of this, a number of new modalities were considered. Spoken in- and output and gaze tracking were selected together with the concept of an interface agent for further investigation. A system was developed to control the interaction of the in- and output modalities, and set of five scenarios were constructed to test the proposed ideas. In these, a number of test subjects used the existing and added modalities in various configurations.

Full Paper

Bibliographic reference.  Bakman, Lau / Blidegn, Mads / Wittrup, Martin / Larsen, Lars Bo / Moeslund, Thomas B. (1998): "Enhancing a WIMP based interface with speech, gaze tracking and agents", In ICSLP-1998, paper 0766.