11th Annual Conference of the International Speech Communication Association

Makuhari, Chiba, Japan
September 26-30. 2010

Multimodal Speaker Diarization Using Oriented Optical Flow Histograms

Mary Tai Knox, Gerald Friedland


Speaker diarization is the task of partitioning an input stream into speaker homogeneous regions, or in other words, to determine ”who spoke when.” While approaches to this problem have traditionally relied entirely on the audio stream, the availability of accompanying video streams in recent diarization corpora has prompted the study of methods based on multimodal audio-visual features. In this work, we propose the use of robust video features based on oriented optical flow histograms. Using the state-of-the art ICSI diarization system, we show that, when combined with standard audio features, these features improve the diarization error rate by 14% percent over an audio only baseline.

Full Paper

Bibliographic reference.  Knox, Mary Tai / Friedland, Gerald (2010): "Multimodal speaker diarization using oriented optical flow histograms", In INTERSPEECH-2010, 290-293.