Overview of the 2017 Spoken CALL Shared Task

Claudia Baur, Cathy Chua, Johanna Gerlach, Manny Rayner, Martin Russell, Helmer Strik, Xizi Wei


We present an overview of a first shared task for spoken CALL. Groups competed on a prompt-response task using English-language data collected, using an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken reply. The task is to accept linguistically correct responses and reject linguistically incorrect ones, to match a gold standard; scoring was performed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, correlations between different possible metrics, and suggestions for a continuation.


 DOI: 10.21437/SLaTE.2017-13

Cite as: Baur, C., Chua, C., Gerlach, J., Rayner, M., Russell, M., Strik, H., Wei, X. (2017) Overview of the 2017 Spoken CALL Shared Task. Proc. 7th ISCA Workshop on Speech and Language Technology in Education, 71-78, DOI: 10.21437/SLaTE.2017-13.


@inproceedings{Baur2017,
  author={Claudia Baur and Cathy Chua and Johanna Gerlach and Manny Rayner and Martin Russell and Helmer Strik and Xizi Wei},
  title={ Overview of the 2017 Spoken CALL Shared Task},
  year=2017,
  booktitle={Proc. 7th ISCA Workshop on Speech and Language Technology in Education},
  pages={71--78},
  doi={10.21437/SLaTE.2017-13},
  url={http://dx.doi.org/10.21437/SLaTE.2017-13}
}