16th Annual Conference of the International Speech Communication Association

Dresden, Germany
September 6-10, 2015

Complete-Linkage Clustering for Voice Activity Detection in Audio and Visual Speech

Houman Ghaemmaghami, David Dean, Shahram Kalantari, Sridha Sridharan, Clinton Fookes

Queensland University of Technology, Australia

We propose a novel technique for conducting robust voice activity detection (VAD) in high-noise recordings. We use Gaussian mixture modeling (GMM) to train two generic models; speech and non-speech. We then score smaller segments of a given (unseen) recording against each of these GMMs to obtain two respective likelihood scores for each segment. These scores are used to compute a dissimilarity measure between pairs of segments and to carry out complete-linkage clustering of the segments into speech and non-speech clusters. We compare the accuracy of our method against state-of-the-art and standardised VAD techniques to demonstrate an absolute improvement of 15% in half-total error rate (HTER) over the best performing baseline system and across the QUT-NOISE-TIMIT database. We then apply our approach to the Audio-Visual Database of American English (AVDBAE) to demonstrate the performance of our algorithm in using visual, audio-visual or a proposed fusion of these features.

Full Paper

Bibliographic reference.  Ghaemmaghami, Houman / Dean, David / Kalantari, Shahram / Sridharan, Sridha / Fookes, Clinton (2015): "Complete-linkage clustering for voice activity detection in audio and visual speech", In INTERSPEECH-2015, 2292-2296.