We present an overview of the third edition of the Spoken CALL Shared Task. Groups competed on a prompt-response task using English-language data collected, through an online CALL game, from Swiss German teens in their second and third years of learning English. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with "linguistically correct" defined by a gold standard derived from human annotations. Scoring was performed using a metric based on the idea of maximising the ratios correct-accept-rate/false-accept-rate and correct-reject-rate/false-reject-rate. The third edition received sixteen entries, with the best score substantially improving on last year's edition of the task. We analyse factors which make it difficult to label items correctly, concluding that, as in the previous edition, good speech recognition is most important. Finally, we suggest a strategy for continuing the task.
Cite as: Baur, C., Caines, A., Chua, C., Gerlach, J., Qian, M., Rayner, M., Russell, M., Strik, H., Wei, X. (2019) Overview of the 2019 Spoken CALL Shared Task. Proc. 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019), 1-5, doi: 10.21437/SLaTE.2019-1
@inproceedings{baur19_slate,
author={Claudia Baur and Andrew Caines and Cathy Chua and Johanna Gerlach and Mengjie Qian and Manny Rayner and Martin Russell and Helmer Strik and Xizi Wei},
title={{Overview of the 2019 Spoken CALL Shared Task}},
year=2019,
booktitle={Proc. 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019)},
pages={1--5},
doi={10.21437/SLaTE.2019-1}
}