Center for Human Modeling and Simulation

Posterior Regularization for Structured Latent Varaible Models

Kuzman Ganchev, University of Pennsylvania
João Graça, L2F INESC-ID
Ben Taskar, University of Pennsylvania
Jennifer Gillenwater, University of Pennsylvania

Document Type Journal Article


We present posterior regularization, a probabilistic framework for structured, weakly supervised learning. Our framework efficiently incorporates indirect supervision via constraints on posterior distributions of probabilistic models with latent variables. Posterior regularization separates model complexity from the complexity of structural constraints it is desired to satisfy. By directly imposing decomposable regularization on the posterior moments of latent variables during learning, we retain the computational efficiency of the unconstrained model while ensuring desired constraints hold in expectation. We present an efficient algorithm for learning with posterior regularization and illustrate its versatility on a diverse set of structural constraints such as bijectivity, symmetry and group sparsity in several large scale experiments, including multi-view learning, cross-lingual dependency grammar induction, unsupervised part-of-speech induction, and bitext word alignment.


Date Posted: 11 July 2012