Multi-Stream Network with Temporal Attention for Environmental Sound Classification

Xinyu Li, Venkata Chebiyyam, Katrin Kirchhoff


Environmental sound classification systems often do not perform robustly across different sound classification tasks and audio signals of varying temporal structures. We introduce a multi-stream convolutional neural network with temporal attention that addresses these problems. The network relies on three input streams consisting of raw audio and spectral features and utilizes a temporal attention function computed from energy changes over time. Training and classification utilizes decision fusion and data augmentation techniques that incorporate uncertainty. We evaluate this network on three commonly used datasets for environmental sound and audio scene classification and achieve new state-of-the-art performance without any changes in network architecture or front-end preprocessing, thus demonstrating better generalizability.


 DOI: 10.21437/Interspeech.2019-3019

Cite as: Li, X., Chebiyyam, V., Kirchhoff, K. (2019) Multi-Stream Network with Temporal Attention for Environmental Sound Classification. Proc. Interspeech 2019, 3604-3608, DOI: 10.21437/Interspeech.2019-3019.


@inproceedings{Li2019,
  author={Xinyu Li and Venkata Chebiyyam and Katrin Kirchhoff},
  title={{Multi-Stream Network with Temporal Attention for Environmental Sound Classification}},
  year=2019,
  booktitle={Proc. Interspeech 2019},
  pages={3604--3608},
  doi={10.21437/Interspeech.2019-3019},
  url={http://dx.doi.org/10.21437/Interspeech.2019-3019}
}