7th International Conference on Spoken Language Processing

September 16-20, 2002
Denver, Colorado, USA

Implementation Testing of a Hybrid Symbolic/Statistical Multimodal Architecture

Edward C. Kaiser, Philip R. Cohen

Oregon Health & Science University, USA

The design and implementation of hybrid symbolic/statistical architectures is a major area of interest in current multimodal system development. Such an architecture attempts to improve multimodal recognition and disambiguation rates by using corpus-based statistics to weight the contributions from various input streams. This is in contrast to current architectures that assume independence between input streams, and combine un-weighted posterior probabilities simply by taking their cross product.

Recently a Members, Teams, Committee (MTC) approach for statistically hybridizing the Quickset multimodal system has been put forward on the basis of strong empirical results in an offline analysis. MTC uses small-dimensional input streams as Members, which in turn are input into various Teams where their conditional weights are trained. The Committee then extracts a decision from the output of the Teams. This paper discusses a fully implemented regression test of MTC within Quickset, and our modification of the approach to use more specific training features. We report a relative decrease in multimodal error rate of 30%.

Full Paper

Bibliographic reference.  Kaiser, Edward C. / Cohen, Philip R. (2002): "Implementation testing of a hybrid symbolic/statistical multimodal architecture", In ICSLP-2002, 173-176.