Modelling the Informativeness of Non-Verbal Cues in Parent-Child Interaction

Mats Wirén, Kristina N. Björkenstam, Robert Östling

Non-verbal cues from speakers, such as eye gaze and hand positions, play an important role in word learning [1]. This is consistent with the notion that for meaning to be reconstructed, acoustic patterns need to be linked to time-synchronous patterns from at least one other modality [2]. In previous studies of a multimodally annotated corpus of parent-child interaction, we have shown that parents interacting with infants at the early word-learning stage (7–9 months) display a large amount of time-synchronous patterns, but that this behaviour tails off with increasing age of the children [3]. Furthermore, we have attempted to quantify the informativeness of the different non-verbal cues, that is, to what extent they actually help to discriminate between different possible referents, and how critical the timing of the cues is [4]. The purpose of this paper is to generalise our earlier model by quantifying informativeness resulting from non-verbal cues occurring both before and after their associated verbal references.

 DOI: 10.21437/Interspeech.2017-1143

Cite as: Wirén, M., Björkenstam, K.N., Östling, R. (2017) Modelling the Informativeness of Non-Verbal Cues in Parent-Child Interaction. Proc. Interspeech 2017, 2203-2207, DOI: 10.21437/Interspeech.2017-1143.

  author={Mats Wirén and Kristina N. Björkenstam and Robert Östling},
  title={Modelling the Informativeness of Non-Verbal Cues in Parent-Child Interaction},
  booktitle={Proc. Interspeech 2017},