STiLL - Speech Technology in Language Learning May 25-27, 1998 Marholmen, Sweden

Intelligent Animated Agents for Interactive Language Training

Ron Cole (1), Tim Carmell (1), Pom Connors (1), Mike Macon (1), Johan Wouters (1), Jacques de Villiers (1), Alice Tarachow (1), Dominic Massaro (2), Michael Cohen (2), Jonas Beskow (2), Jie Yang (3), Uwe Meier (3), Alex Waibel (3), Pat Stone (4), George Fortier (4), Alice Davis (4), Chris Soland (4)

(1) Center for Spoken Language Understanding, Oregon Graduate Institute, Portland, OR, USA
(2) Perceptual Science Laboratory, University of California, Santa Cruz, CA, USA
(3) Interactive Systems Laboratory, Carnegie Mellon University, Pittsburgh, PA, USA
(4) Tucker-Maxon Oral School, Portland, OR, USA

This report describes a three-year project, now eight months old, to develop interactive learning tools for language training with profoundly deaf children. The tools combine four key technologies: speech recognition, developed at the Oregon Graduate Institute; speech synthesis, developed at the University of Edinburgh and modified at OGI; facial animation, developed at University of California, Santa Cruz; and face tracking and speech reading, developed at Carnegie Mellon University. These technologies are being combined to create an intelligent conversational agent; a three-dimensional face that produces and understands auditory and visual speech. The agent has been incorporated into the CSLU Toolkit, a software environment for developing and researching spoken language systems. We describe our experiences in bringing interactive learning tools to classrooms at the Tucker-Maxon Oral School in Portland, Oregon, and the technological advances that are required for this project to succeed.

Full Paper

Bibliographic reference.  Cole, Ron / Carmell, Tim / Connors, Pom / Macon, Mike / Wouters, Johan / Villiers, Jacques de / Tarachow, Alice / Massaro, Dominic / Cohen, Michael / Beskow, Jonas / Yang, Jie / Meier, Uwe / Waibel, Alex / Stone, Pat / Fortier, George / Davis, Alice / Soland, Chris (1998): "Intelligent animated agents for interactive language training", In STiLL-1998, 163-166.