ISCA Archive Interspeech 2013
ISCA Archive Interspeech 2013

Detecting overlapping speech with long short-term memory recurrent neural networks

Jürgen T. Geiger, Florian Eyben, Björn Schuller, Gerhard Rigoll

Detecting segments of overlapping speech (when two or more speakers are active at the same time) is a challenging problem. Previously, mostly HMM-based systems have been used for overlap detection, employing various different audio features. In this work, we propose a novel overlap detection system using Long Short-Term Memory (LSTM) recurrent neural networks. LSTMs are used to generate framewise overlap predictions which are applied for overlap detection. Furthermore, a tandem HMM-LSTM system is obtained by adding LSTM predictions to the HMM feature set. Experiments with the AMI corpus show that overlap detection performance of LSTMs is comparable to HMMs. The combination of HMMs and LSTMs improves overlap detection by achieving higher recall.


doi: 10.21437/Interspeech.2013-27

Cite as: Geiger, J.T., Eyben, F., Schuller, B., Rigoll, G. (2013) Detecting overlapping speech with long short-term memory recurrent neural networks. Proc. Interspeech 2013, 1668-1672, doi: 10.21437/Interspeech.2013-27

@inproceedings{geiger13_interspeech,
  author={Jürgen T. Geiger and Florian Eyben and Björn Schuller and Gerhard Rigoll},
  title={{Detecting overlapping speech with long short-term memory recurrent neural networks}},
  year=2013,
  booktitle={Proc. Interspeech 2013},
  pages={1668--1672},
  doi={10.21437/Interspeech.2013-27}
}