Progressive Neural Networks for Transfer Learning in Emotion Recognition

John Gideon, Soheil Khorram, Zakaria Aldeneh, Dimitrios Dimitriadis, Emily Mower Provost


Many paralinguistic tasks are closely related and thus representations learned in one domain can be leveraged for another. In this paper, we investigate how knowledge can be transferred between three paralinguistic tasks: speaker, emotion, and gender recognition. Further, we extend this problem to cross-dataset tasks, asking how knowledge captured in one emotion dataset can be transferred to another. We focus on progressive neural networks and compare these networks to the conventional deep learning method of pre-training and fine-tuning. Progressive neural networks provide a way to transfer knowledge and avoid the forgetting effect present when pre-training neural networks on different tasks. Our experiments demonstrate that: (1) emotion recognition can benefit from using representations originally learned for different paralinguistic tasks and (2) transfer learning can effectively leverage additional datasets to improve the performance of emotion recognition systems.


 DOI: 10.21437/Interspeech.2017-1637

Cite as: Gideon, J., Khorram, S., Aldeneh, Z., Dimitriadis, D., Provost, E.M. (2017) Progressive Neural Networks for Transfer Learning in Emotion Recognition. Proc. Interspeech 2017, 1098-1102, DOI: 10.21437/Interspeech.2017-1637.


@inproceedings{Gideon2017,
  author={John Gideon and Soheil Khorram and Zakaria Aldeneh and Dimitrios Dimitriadis and Emily Mower Provost},
  title={Progressive Neural Networks for Transfer Learning in Emotion Recognition},
  year=2017,
  booktitle={Proc. Interspeech 2017},
  pages={1098--1102},
  doi={10.21437/Interspeech.2017-1637},
  url={http://dx.doi.org/10.21437/Interspeech.2017-1637}
}