16th Annual Conference of the International Speech Communication Association

Dresden, Germany
September 6-10, 2015

A Hybrid Dynamic Time Warping-Deep Neural Network Architecture for Unsupervised Acoustic Modeling

Roland Thiollière, Ewan Dunbar, Gabriel Synnaeve, Maarten Versteegh, Emmanuel Dupoux

ENS, France

We report on an architecture for the unsupervised discovery of talker-invariant subword embeddings. It is made out of two components: a dynamic-time warping based spoken term discovery (STD) system and a Siamese deep neural network (DNN). The STD system clusters word-sized repeated fragments in the acoustic streams while the DNN is trained to minimize the distance between time aligned frames of tokens of the same cluster, and maximize the distance between tokens of different clusters. We use additional side information regarding the average duration of phonemic units, as well as talker identity tags. For evaluation we use the datasets and metrics of the Zero Resource Speech Challenge. The model shows improvement over the baseline in subword unit modeling.

Full Paper

Bibliographic reference.  Thiollière, Roland / Dunbar, Ewan / Synnaeve, Gabriel / Versteegh, Maarten / Dupoux, Emmanuel (2015): "A hybrid dynamic time warping-deep neural network architecture for unsupervised acoustic modeling", In INTERSPEECH-2015, 3179-3183.