Differential topic models
Loading...
Date
Authors
Chen, Changyou
Buntine, Wray
Ding, Nan
Xie, Lexing
Du, Lan
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Abstract
In applications we may want to compare different document collections: they could have shared content but also different
and unique aspects in particular collections. This task has been called comparative text mining or cross-collection modeling. We
present a differential topic model for this application that models both topic differences and similarities. For this we use hierarchical
Bayesian nonparametric models. Moreover, we found it was important to properly model power-law phenomena in topic-word
distributions and thus we used the full Pitman-Yor process rather than just a Dirichlet process. Furthermore, we propose the
transformed Pitman-Yor process (TPYP) to incorporate prior knowledge such as vocabulary variations in different collections into the
model. To deal with the non-conjugate issue between model prior and likelihood in the TPYP, we thus propose an efficient sampling
algorithm using a data augmentation technique based on the multinomial theorem. Experimental results show the model discovers
interesting aspects of different collections. We also show the proposed MCMC based algorithm achieves a dramatically reduced test
perplexity compared to some existing topic models. Finally, we show our model outperforms the state-of-the-art for document
classification/ideology prediction on a number of text collections.
Description
Citation
Collections
Source
IEEE Transactions on Pattern Analysis and Machine Intelligence
Type
Book Title
Entity type
Access Statement
License Rights
Restricted until
Downloads
File
Description