Building a Multimodal Lexicon: Lessons from Infants' Learning of Body Part Words

Rana Abu-Zhaya, Amanda Seidl, Ruth Tincoff, Alejandrina Cristia


Human children outperform artificial learners because the former quickly acquire a multimodal, syntactically informed, and ever-growing lexicon with little evidence. Most of this lexicon is unlabelled and processed with unsupervised mechanisms, leading to robust and generalizable knowledge. In this paper, we summarize results related to 4-month-olds’ learning of body part words. In addition to providing direct experimental evidence on some of the Workshop’s assumptions, we suggest several avenues of research that may be useful to those developing and testing artificial learners. A first set of studies using a controlled laboratory learning paradigm shows that human infants learn better from tactile-speech than visual-speech co-occurrences, suggesting that the signal/modality should be considered when designing and exploiting multimodal learning tasks. A series of observational studies document the ways in which parents naturally structure the multimodal information they provide for infants, which probably happens in lexically specific ways. Finally, our results suggest that 4-month-olds can pick up on co-occurrences between words and specific touch locations (a prerequisite of learning an association between a body part word and the referent on the child’s own body) after very brief exposures, which we interpret as most compatible with unsupervised predictive models of learning.


 DOI: 10.21437/GLU.2017-4

Cite as: Abu-Zhaya, R., Seidl, A., Tincoff, R., Cristia, A. (2017) Building a Multimodal Lexicon: Lessons from Infants' Learning of Body Part Words. Proc. GLU 2017 International Workshop on Grounding Language Understanding, 18-21, DOI: 10.21437/GLU.2017-4.


@inproceedings{Abu-Zhaya2017,
  author={Rana Abu-Zhaya and Amanda Seidl and Ruth Tincoff and Alejandrina Cristia},
  title={Building a Multimodal Lexicon: Lessons from Infants' Learning of Body Part Words},
  year=2017,
  booktitle={Proc. GLU 2017 International Workshop on Grounding Language Understanding},
  pages={18--21},
  doi={10.21437/GLU.2017-4},
  url={http://dx.doi.org/10.21437/GLU.2017-4}
}