8th Annual Conference of the International Speech Communication Association

Antwerp, Belgium
August 27-31, 2007

Intercoder Reliability in Annotating Complex Disfluencies

Peter A. Heeman (1), Andy McMillin (2), J. Scott Yaruss (3)

(1) Oregon Health & Science University, USA
(2) Hearing & Speech Institute, USA
(3) University of Pittsburgh, USA

In previous work, we presented an annotation scheme that can describe complex disfluencies. In this paper, we first show the prevalence of complex disfluencies and illustrate the types of distinctions that our scheme allows. Second, we present an annotation tool that allows the scheme to be easily applied. Third, we present the results of a reliability study in annotating complex disfluencies with the annotation tool. We find that subjects, even with a minimal amount of training, achieve high intercoder agreement. This work will help pave the way for speech recognizers to precisely model the structure of disfluencies, both for understanding conversational speech of non-stutterers and for assessing stuttering severity.

Full Paper

Bibliographic reference.  Heeman, Peter A. / McMillin, Andy / Yaruss, J. Scott (2007): "Intercoder reliability in annotating complex disfluencies", In INTERSPEECH-2007, 954-957.