15th Annual Conference of the International Speech Communication Association

September 14-18, 2014

Classification of Cognitive Load from Speech Using an i-Vector Framework

Maarten Van Segbroeck, Ruchir Travadi, Colin Vaz, Jangwon Kim, Matthew P. Black, Alexandros Potamianos, Shrikanth S. Narayanan

University of Southern California, USA

The goal in this work is to automatically classify speakers' level of cognitive load (low, medium, high) from a standard battery of reading tasks requiring varying levels of working memory. This is a challenging machine learning problem because of the inherent difficulty in defining/measuring cognitive load and due to intra-/inter-speaker differences in how their effects are manifested in behavioral cues. We experimented with a number of static and dynamic features extracted directly from the audio signal (prosodic, spectral, voice quality) and from automatic speech recognition hypotheses (lexical information, speaking rate). Our approach to classification addressed the wide variability and heterogeneity through speaker normalization and by adopting an i-vector framework that affords a systematic way to factorize the multiple sources of variability.

Full Paper

Bibliographic reference.  Segbroeck, Maarten Van / Travadi, Ruchir / Vaz, Colin / Kim, Jangwon / Black, Matthew P. / Potamianos, Alexandros / Narayanan, Shrikanth S. (2014): "Classification of cognitive load from speech using an i-vector framework", In INTERSPEECH-2014, 751-755.