ISCA Archive ICSLP 2000
ISCA Archive ICSLP 2000

Labeler agreement in transcribing korean intonation with K-toBI

Sun-Ah Jun, Sook-Hyang Lee, Keeho Kim, Yong-Ju Lee

This paper reports labeler agreement in the transcription of Korean prosody using Korean ToBI (K-ToBI). Twenty utterances representing five different types of speech were produced by 18 speakers and transcribed by 21 labelers differing in their levels of experience with K-ToBI. Following the stringent metric used for English ToBI evaluation [14,12], consistency was measured in terms of the number of transcriber pairs agreeing on the labeling of each particular word. The results show that for tonal transcriptions of the 32,130 transcriber-pair-words, agreement was 77% for the type of boundaries at the end of each word (i.e., word, AP, or IP), 78% for AP boundaries, and 91% for IP boundaries. For break indices, the agreement score for exact matching in the labeling was 59%, 69% when relaxing the presence/absence of diacritics, and 99% when relaxing within +/-1 level. In sum, the data confirm that the conventions of K-ToBI are adequate, easy to learn, and can be reliably used for research in Korean prosody and for large-scale prosodic annotation in speech databases.


Cite as: Jun, S.-A., Lee, S.-H., Kim, K., Lee, Y.-J. (2000) Labeler agreement in transcribing korean intonation with K-toBI. Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000), vol. 3, 211-214

@inproceedings{jun00_icslp,
  author={Sun-Ah Jun and Sook-Hyang Lee and Keeho Kim and Yong-Ju Lee},
  title={{Labeler agreement in transcribing korean intonation with K-toBI}},
  year=2000,
  booktitle={Proc. 6th International Conference on Spoken Language Processing (ICSLP 2000)},
  pages={vol. 3, 211-214}
}