Joint covariate selection and joint subspace selection for multiple classification problems

Loading...
Thumbnail Image
Penn collection
Lab Papers (GRASP)
Degree type
Discipline
Subject
Variable selection
Subspace selection
Lasso
Group Lasso
Regularization path
Supervised dimensionality reduction
Multitask learning
Block norm
Trace norm
Random projections
Funder
Grant number
License
Copyright date
Distributor
Related resources
Author
Obozinski, Guillaume
Jordan, Michael I
Contributor
Abstract

We address the problem of recovering a common set of covariates that are relevant simultaneously to several classification problems. By penalizing the sum of ℓ2-norms of the blocks of coefficients associated with each covariate across different classification problems, similar sparsity patterns in all models are encouraged. To take computational advantage of the sparsity of solutions at high regularization levels, we propose a blockwise path-following scheme that approximately traces the regularization path. As the regularization coefficient decreases, the algorithm maintains and updates concurrently a growing set of covariates that are simultaneously active for all problems. We also show how to use random projections to extend this approach to the problem of joint subspace selection, where multiple predictors are found in a common low-dimensional subspace. We present theoretical results showing that this random projection approach converges to the solution yielded by trace-norm regularization. Finally, we present a variety of experimental results exploring joint covariate selection and joint subspace selection, comparing the path-following approach to competing algorithms in terms of prediction accuracy and running time.

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2009-01-14
Journal title
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Postprint version. Published in: Joint covariate selection and joint subspace selection for multiple classification problems, G. Obozinski, B.Taskar, M.I. Jordan, Statistics and Computing, Jan. 2009. The original publication is available at www.springerlink.com. Publisher URL: http://dx.doi.org/10.1007/s11222-008-9111-x
Recommended citation
Collection