A Coupled Hidden Markov Model (CHMM) is proposed in this paper to perform multimodal speech recognition using myoelectric signals (MES) from the muscles of vocal articulation. MES signals are immune to noise, and words that are acoustically similar manifest distinctly in MES. Hence, they would effectively complement the acoustic data in a multimodal speech recognition system. Research in Audio-Visual Speech Recognition has shown that CHMMs model the asynchrony between different data streams effectively. Hence, we propose CHMM for multimodal speech recognition using audio and MES as the two data streams. Our experiments indicate that the multimodal CHMM system significantly outperforms the audio only system at different SNRs. We have also provided a comparison between different features for MES and have found that wavelet features provide the best results.
Cite as: Kumaran, R.S., Narayanan, K., Gowdy, J.N. (2005) Myoelectric signals for multimodal speech recognition. Proc. Interspeech 2005, 1189-1192, doi: 10.21437/Interspeech.2005-365
@inproceedings{kumaran05_interspeech,
author={Raghunandan S. Kumaran and Karthik Narayanan and John N. Gowdy},
title={{Myoelectric signals for multimodal speech recognition}},
year=2005,
booktitle={Proc. Interspeech 2005},
pages={1189--1192},
doi={10.21437/Interspeech.2005-365}
}