12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

Crowdsourcing for Word Recognition in Noise

Martin Cooke (1), Jon Barker (2), Maria Luisa Garcia Lecumberri (3), Krzysztof Wasilewski (2)

(1) Ikerbasque, Spain
(2) University of Sheffield, UK
(3) Universidad del País Vasco, Spain

Access to large samples of listeners is an appealing prospect for speech perception researchers, but lack of control over key factors such as listeners' linguistic backgrounds and quality of stimulus delivery is a formidable barrier to the application of crowdsourcing. We describe the outcome of a web-based listening experiment designed to discover consistent confusions amongst words presented in noise, alongside an identical task carried out using traditional laboratory methods. Web listeners were graded based on information they provided as well as via their responses to tokens recognised robustly by a majority of participants. While overall word identification scores even for the best-performing web subset were well below those obtained in the laboratory, word confusions with high levels of cross-listener agreement were obtained nevertheless, suggesting that focused application of crowdsourcing in speech perception can provide useful data for scientific analysis.

Full Paper

Bibliographic reference.  Cooke, Martin / Barker, Jon / Lecumberri, Maria Luisa Garcia / Wasilewski, Krzysztof (2011): "Crowdsourcing for word recognition in noise", In INTERSPEECH-2011, 3049-3052.