Identifying Input Features for Development of Real-Time Translation of Neural Signals to Text

Janaki Sheth, Ariel Tankus, Michelle Tran, Lindy Comstock, Itzhak Fried, William Speier


One of the main goals in Brain-Computer Interface (BCI) research is to help patients with faltering communication abilities due to neurodegenerative diseases produce text or speech output using their neural recordings. However, practical implementation of such a system has proven difficult due to limitations in the speed, accuracy, and training time of existing interfaces. In this paper, we contribute to this endeavour by isolating appropriate input features from speech-producing neural signals that will feed into a machine learning classifier to identify target phonemes. Analysing data from six subjects, we discern frequency bands that encapsulate differential information regarding production of vowels and consonants broadly, and more specifically nasals and semivowels. Subsequent spatial localization analysis reveals the underlying cortical regions responsible for different phoneme categories. Anatomical locations along with their respective frequency bands act as prospective feature sets for machine learning classifiers. We demonstrate this classification ability in a preliminary language reconstruction task and show an average word classification accuracy of 30.6% (p<0.001).


 DOI: 10.21437/Interspeech.2019-3092

Cite as: Sheth, J., Tankus, A., Tran, M., Comstock, L., Fried, I., Speier, W. (2019) Identifying Input Features for Development of Real-Time Translation of Neural Signals to Text. Proc. Interspeech 2019, 869-873, DOI: 10.21437/Interspeech.2019-3092.


@inproceedings{Sheth2019,
  author={Janaki Sheth and Ariel Tankus and Michelle Tran and Lindy Comstock and Itzhak Fried and William Speier},
  title={{Identifying Input Features for Development of Real-Time Translation of Neural Signals to Text}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={869--873},
  doi={10.21437/Interspeech.2019-3092},
  url={http://dx.doi.org/10.21437/Interspeech.2019-3092}
}