ISCA International Workshop on Speech and Language Technology in Education (SLaTE 2011)

Venice, Italy
August 24-26, 2011

How Many Labellers Revisited – Naïves, Experts, and Real Experts

Florian Hönig, Anton Batliner, Elmar Nöth

Pattern Recognition Lab, Universität Erlangen-Nürnberg, Germany

A database of non-native German productions was annotated by three different groups: by experts using detailed, localised labels as well as coarse, global labels, and by phoneticians and na¨ýve subjects, using the same coarse global labels. For the detailed annotation, segmental and supra-segmental labels were given segment-based and word-based. The global annotation consisted of a turn-based assessment of intelligibility, nonnative accent, melody, and rhythm. Moreover, we use a large, specialised prosodic feature vector for modelling native vs. nonnative speech. We study relationships between detailed and global labels, analyse the quality of expert and na¨ýve labellers, and present an automatic system for predicting a speaker’s score for the global labels.
Index Terms. non-native prosody, speech melody, rhythm, inter-labeller agreement, regression system, performance model, experts vs. naïve judges

Full Paper

Bibliographic reference.  Hönig, Florian / Batliner, Anton / Nöth, Elmar (2011): "How many labellers revisited – naïves, experts, and real experts", In SLaTE-2011, 137-140.