Date of this Version
Journal of the American Statistical Association
Despite the wide adoption of spike-and-slab methodology for Bayesian variable selection, its potential for penalized likelihood estimation has largely been overlooked. In this article, we bridge this gap by cross-fertilizing these two paradigms with the Spike-and-Slab LASSO procedure for variable selection and parameter estimation in linear regression. We introduce a new class of self-adaptive penalty functions that arise from a fully Bayes spike-and-slab formulation, ultimately moving beyond the separable penalty framework. A virtue of these nonseparable penalties is their ability to borrow strength across coordinates, adapt to ensemble sparsity information and exert multiplicity adjustment. The Spike-and-Slab LASSO procedure harvests efficient coordinate-wise implementations with a path-following scheme for dynamic posterior exploration. We show on simulated data that the fully Bayes penalty mimics oracle performance, providing a viable alternative to cross-validation. We develop theory for the separable and nonseparable variants of the penalty, showing rate-optimality of the global mode as well as optimal posterior concentration when p > n. Supplementary materials for this article are available online.
This is an Accepted Manuscript of an article published by Taylor & Francis in the Journal of the American Statistical Association on 16 Dec 2016, available online: http://dx.doi.org/10.1080/01621459.2016.1260469
high-dimensional regression, LASSO, penalized likelihood, posterior concentration, spike-and-slab, variable selection
Ročková, V., & George, E. I. (2016). The Spike-and-Slab LASSO. Journal of the American Statistical Association, 113 (521), 431-444. http://dx.doi.org/10.1080/01621459.2016.1260469
Date Posted: 25 October 2018
This document has been peer reviewed.