Posterior Regularization for Structured Latent Varaible Models

Loading...
Thumbnail Image
Penn collection
Departmental Papers (CIS)
Degree type
Discipline
Subject
Computer Sciences
Funder
Grant number
License
Copyright date
Distributor
Related resources
Author
Ganchev, Kuzman
Graça, João
Gillenwater, Jennifer
Contributor
Abstract

We present posterior regularization, a probabilistic framework for structured, weakly supervised learning. Our framework efficiently incorporates indirect supervision via constraints on posterior distributions of probabilistic models with latent variables. Posterior regularization separates model complexity from the complexity of structural constraints it is desired to satisfy. By directly imposing decomposable regularization on the posterior moments of latent variables during learning, we retain the computational efficiency of the unconstrained model while ensuring desired constraints hold in expectation. We present an efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment.

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2010-07-01
Journal title
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Posterior Regularization for Structured Latent Variable Models (http://www.cis.upenn.edu/%7Etaskar/pubs/pr_jmlr10.pdf), K. Ganchev (http://www.seas.upenn.edu/%7Ekuzman/), J. Graca (http://www.cis.upenn.edu/%7Egraca/), J. Gillenwater (http://www.seas.upenn.edu/%7Ejengi") and B. Taskar, Journal of Machine Learning Research (JMLR) (http://jmlr.csail.mit.edu/), July 2010. Copyright held by the authors.
Recommended citation
Collection