ISCA Archive ICSLP 2000
ISCA Archive ICSLP 2000

Multimodal signal processing in naturalistic noisy environments

Sharon Oviatt

When a system must process spoken language in natural environments that involve different types and levels of noise, the problem of supporting robust recognition is a very difficult one. In the present studies, over 2,600 multimodal utterances were collected during both mobile and stationary use of a multimodal pen/voice system. The results confirmed that multimodal signal processing supports significantly improved robustness over spoken language processing alone, with the largest improvement during mobile use. The multimodal architecture decreased the spoken language error rate by 19-35%. In addition, data collected on a command-by-command basis while users were mobile emphasized the adverse impact of usersÂ’ Lombard adaptation on system processing, even when a noise-canceling microphone was used. Implications of these findings are discussed for improving the reliability and stability of spoken language processing in mobile environments.


Cite as: Oviatt, S. (2000) Multimodal signal processing in naturalistic noisy environments. Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), vol. 2, 696-699

@inproceedings{oviatt00_icslp,
  author={Sharon Oviatt},
  title={{Multimodal signal processing in naturalistic noisy environments}},
  year=2000,
  booktitle={Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000)},
  pages={vol. 2, 696-699}
}