Communication with Speech and Gestures: Applications of Recurrent Neural Networks to Robot Language Learning

Alexandre Antunes, Gabriella Pizzuto, Angelo Cangelosi


Recurrent neural networks have recently shown significant potential in different language applications, ranging from natural language processing to language modelling. This paper introduces a research effort to use such networks to develop and evaluate natural language acquisition on a humanoid robot. Here, the problem is twofold. First, the focus will be put on using the gesture-word combination stage observed in infants to transition from single to multi-word utterances. Secondly, research will be carried out in the domain of connecting action learning with language learning. In the former, the long-short term memory architecture will be implemented, whilst in the latter multiple time-scale recurrent neural networks will be used. This will allow for comparison between the two architectures, whilst highlighting the strengths and shortcomings of both with respect to the language learning problem. Here, the main research efforts, challenges and expected outcomes are described.


 DOI: 10.21437/GLU.2017-1

Cite as: Antunes, A., Pizzuto, G., Cangelosi, A. (2017) Communication with Speech and Gestures: Applications of Recurrent Neural Networks to Robot Language Learning. Proc. GLU 2017 International Workshop on Grounding Language Understanding, 4-7, DOI: 10.21437/GLU.2017-1.


@inproceedings{Antunes2017,
  author={Alexandre Antunes and Gabriella Pizzuto and Angelo Cangelosi},
  title={Communication with Speech and Gestures: Applications of Recurrent Neural Networks to Robot Language Learning},
  year=2017,
  booktitle={Proc. GLU 2017 International Workshop on Grounding Language Understanding},
  pages={4--7},
  doi={10.21437/GLU.2017-1},
  url={http://dx.doi.org/10.21437/GLU.2017-1}
}