Speech and Language Technology in Education (SLaTE 2013)
In this paper we present a novel approach for automatically tracking the reading progress using a combination of eye-gaze tracking and speech recognition. The two are fused by first generating word probabilities based on eye-gaze information and then using these probabilities to augment the language model probabilities during speech recognition. Experimental results on a small dataset show that the tracking error rate of the system using only speech recognition is 37.9% whereas the tracking error rate for the system that incorporates eye-gaze tracking into the speech recognizer is 35.8%.
Index Terms: automatic reading tutor, eye-gaze tracking, speech recognition
Bibliographic reference. Rasmussen, Morten Højfeldt / Tan, Zheng-Hua (2013): "Fusing eye-gaze and speech recognition for tracking in an automatic reading tutor a step in the right direction?", In SLaTE-2013, 112-115.