Sixth International Conference on Spoken Language Processing
(ICSLP 2000)

Beijing, China
October 16-20, 2000

Labeler Agreement in Transcribing Korean Intonation with K-ToBI

Sun-Ah Jun (1), Sook-Hyang Lee (2), Keeho Kim (3), Yong-Ju Lee (4)

(1) Dept. of Linguistics, UCLA, USA; (2) Division of English & Chinese, Wonkwang Univ., Korea; (3) Dept. of English Language & Literature, Korea Univ., Korea; (4) Dept. of Electrical, Electronics & Information Engineering, Wonkwang Univ., Korea

This paper reports labeler agreement in the transcription of Korean prosody using Korean ToBI (K-ToBI). Twenty utterances representing five different types of speech were produced by 18 speakers and transcribed by 21 labelers differing in their levels of experience with K-ToBI. Following the stringent metric used for English ToBI evaluation [14,12], consistency was measured in terms of the number of transcriber pairs agreeing on the labeling of each particular word. The results show that for tonal transcriptions of the 32,130 transcriber-pair-words, agreement was 77% for the type of boundaries at the end of each word (i.e., word, AP, or IP), 78% for AP boundaries, and 91% for IP boundaries. For break indices, the agreement score for exact matching in the labeling was 59%, 69% when relaxing the presence/absence of diacritics, and 99% when relaxing within +/-1 level. In sum, the data confirm that the conventions of K-ToBI are adequate, easy to learn, and can be reliably used for research in Korean prosody and for large-scale prosodic annotation in speech databases.


Full Paper

Bibliographic reference.  Jun, Sun-Ah / Lee, Sook-Hyang / Kim, Keeho / Lee, Yong-Ju (2000): "Labeler agreement in transcribing korean intonation with K-toBI", In ICSLP-2000, vol.3, 211-214.