Toward Remote Patient Monitoring of Speech, Video, Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology

Vikram Ramanarayanan, Oliver Roesler, Michael Neumann, David Pautler, Doug Habberstad, Andrew Cornish, Hardik Kothare, Vignesh Murali, Jackson Liscombe, Dirk Schnelle-Walka, Patrick Lange, David Suendermann-Oeft


We demonstrate a multimodal conversational platform for remote patient diagnosis and monitoring. The platform engages patients in an interactive dialog session and automatically computes metrics relevant to speech acoustics and articulation, oro-motor and oro-facial movement, cognitive function and respiratory function. The dialog session includes a selection of exercises that have been widely used in both speech language pathology research as well as clinical practice — an oral motor exam, sustained phonation, diadochokinesis, read speech, spontaneous speech, spirometry, picture description, emotion elicitation and other cognitive tasks. Finally, the system automatically computes speech, video, cognitive and respiratory biomarkers that have been shown to be useful in capturing various aspects of speech motor function and neurological health and visualizes them in a user-friendly dashboard.


Cite as: Ramanarayanan, V., Roesler, O., Neumann, M., Pautler, D., Habberstad, D., Cornish, A., Kothare, H., Murali, V., Liscombe, J., Schnelle-Walka, D., Lange, P., Suendermann-Oeft, D. (2020) Toward Remote Patient Monitoring of Speech, Video, Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology. Proc. Interspeech 2020, 492-493.


@inproceedings{Ramanarayanan2020,
  author={Vikram Ramanarayanan and Oliver Roesler and Michael Neumann and David Pautler and Doug Habberstad and Andrew Cornish and Hardik Kothare and Vignesh Murali and Jackson Liscombe and Dirk Schnelle-Walka and Patrick Lange and David Suendermann-Oeft},
  title={{Toward Remote Patient Monitoring of Speech, Video, Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology}},
  year=2020,
  booktitle={Proc. Interspeech 2020},
  pages={492--493}
}