Learning Document Representations Using Subspace Multinomial Model

Santosh Kesiraju, Lukáš Burget, Igor Szőke, Jan Černocký


Subspace multinomial model (SMM) is a log-linear model and can be used for learning low dimensional continuous representation for discrete data. SMM and its variants have been used for speaker verification based on prosodic features and phonotactic language recognition. In this paper, we propose a new variant of SMM that introduces sparsity and call the resulting model as ℓ1 SMM. We show that ℓ1 SMM can be used for learning document representations that are helpful in topic identification or classification and clustering tasks. Our experiments in document classification show that SMM achieves comparable results to models such as latent Dirichlet allocation and sparse topical coding, while having a useful property that the resulting document vectors are Gaussian distributed.


DOI: 10.21437/Interspeech.2016-1634

Cite as

Kesiraju, S., Burget, L., Szőke, I., Černocký, J. (2016) Learning Document Representations Using Subspace Multinomial Model. Proc. Interspeech 2016, 700-704.

Bibtex
@inproceedings{Kesiraju+2016,
author={Santosh Kesiraju and Lukáš Burget and Igor Szőke and Jan Černocký},
title={Learning Document Representations Using Subspace Multinomial Model},
year=2016,
booktitle={Interspeech 2016},
doi={10.21437/Interspeech.2016-1634},
url={http://dx.doi.org/10.21437/Interspeech.2016-1634},
pages={700--704}
}