ISCA International Workshop on Speech and Language Technology in Education (SLaTE 2011)

Venice, Italy
August 24-26, 2011

Evaluating Tracking Accuracy of an Automatic Reading Tutor

Morten Højfeldt Rasmussen (1), Jack Mostow (2), Zheng-Hua Tan (1), Børge Lindberg (1), Yuanpeng Li (2)

(1) Department of Electronic Systems, Aalborg University, Denmark
(2) School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA

In automatic reading tutoring, tracking is the process of automatically following a reader through a given target text. When developing tracking algorithms, a measure of the tracking accuracy – how often a spoken word is aligned to the right target text word position – is needed in order to evaluate performance and compare different algorithms. This paper presents a framework for determining the observed tracking error rate. The proposed framework is used to evaluate three tracking strategies: A) follow the reader to whichever word he/she jumps to in the text, B) follow the reader monotonically from left to right ignoring word skips and regressions (going back to a previous text word), and C) the same as B but allowing isolated word skips. Observed tracking error rate for each of the three tracking strategies is: A: 53%, B: 56%, and C: 47%, on 1883 utterances from 25 children.
Index Terms. Automatic Reading Tutor, Tracking Speech, Tracking Error Rate

Full Paper

Bibliographic reference.  Rasmussen, Morten Højfeldt / Mostow, Jack / Tan, Zheng-Hua / Lindberg, Børge / Li, Yuanpeng (2011): "Evaluating tracking accuracy of an automatic reading tutor", In SLaTE-2011, 17-20.