SLaTE 2015 - Workshop on Speech and Language Technology in Education

Leipzig, Germany
September 4-5, 2015

Using Karaoke to Enhance Reading While Listening: Impact on Word Memorization and Eye Movements

Emilie Gerbier (1), Gérard Bailly (1), Marie-Line Bosse (2)

(1) GIPSA-Lab; (2) LPNC
Univ. Grenoble Alpes/CNRS, Grenoble, France

This article reports the use of a karaoke technique to drive the visual attention span (VAS) of subjects reading a text while listening to the text spelled aloud by a reading tutor. We tested the impact of computer-assisted synchronous reading (S+) that emphasizes words when they are uttered, vs. nonsynchronous reading (S-) in a reading while listening (RWL) task. Thirty-five 6th grade pupils read 12 stories, each involving one pseudoword presented four times, and each displayed in either condition. They were then unexpectedly tested on their memory for the orthography and for their acquired semantic knowledge of the pseudowords. Although no benefit was observed in the orthographic task, the synchronous condition significantly boosted the semantic memory by 10 points compared to the nonsynchronous one (28% vs. 17% correct). We also provide some preliminary analysis on the gaze data collected during reading, suggesting differences between both conditions in terms of first fixation duration, fixation position on the word and onset delay relative to the corresponding speech onset.

Full Paper

Bibliographic reference.  Gerbier, Emilie / Bailly, Gérard / Bosse, Marie-Line (2015): "Using Karaoke to enhance reading while listening: impact on word memorization and eye movements", In SLaTE-2015, 59-64.