ISCA Archive SLaTE 2017
ISCA Archive SLaTE 2017

Overview of the 2017 Spoken CALL Shared Task

Claudia Baur, Cathy Chua, Johanna Gerlach, Manny Rayner, Martin Russell, Helmer Strik, Xizi Wei

We present an overview of a first shared task for spoken CALL. Groups competed on a prompt-response task using English-language data collected, using an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken reply. The task is to accept linguistically correct responses and reject linguistically incorrect ones, to match a gold standard; scoring was performed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, correlations between different possible metrics, and suggestions for a continuation.


doi: 10.21437/SLaTE.2017-13

Cite as: Baur, C., Chua, C., Gerlach, J., Rayner, M., Russell, M., Strik, H., Wei, X. (2017) Overview of the 2017 Spoken CALL Shared Task. Proc. 7th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2017), 71-78, doi: 10.21437/SLaTE.2017-13

@inproceedings{baur17_slate,
  author={Claudia Baur and Cathy Chua and Johanna Gerlach and Manny Rayner and Martin Russell and Helmer Strik and Xizi Wei},
  title={{Overview of the 2017 Spoken CALL Shared Task}},
  year=2017,
  booktitle={Proc. 7th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2017)},
  pages={71--78},
  doi={10.21437/SLaTE.2017-13}
}