Statistics Papers

Document Type

Technical Report

Date of this Version

12-16-2016

Publication Source

Journal of the American Statistical Association

Volume

113

Issue

521

Start Page

431

Last Page

444

DOI

10.1080/01621459.2016.1260469

Abstract

Despite the wide adoption of spike-and-slab methodology for Bayesian variable selection, its potential for penalized likelihood estimation has largely been overlooked. In this article, we bridge this gap by cross-fertilizing these two paradigms with the Spike-and-Slab LASSO procedure for variable selection and parameter estimation in linear regression. We introduce a new class of self-adaptive penalty functions that arise from a fully Bayes spike-and-slab formulation, ultimately moving beyond the separable penalty framework. A virtue of these nonseparable penalties is their ability to borrow strength across coordinates, adapt to ensemble sparsity information and exert multiplicity adjustment. The Spike-and-Slab LASSO procedure harvests efficient coordinate-wise implementations with a path-following scheme for dynamic posterior exploration. We show on simulated data that the fully Bayes penalty mimics oracle performance, providing a viable alternative to cross-validation. We develop theory for the separable and nonseparable variants of the penalty, showing rate-optimality of the global mode as well as optimal posterior concentration when p > n. Supplementary materials for this article are available online.

Copyright/Permission Statement

This is an Accepted Manuscript of an article published by Taylor & Francis in the Journal of the American Statistical Association on 16 Dec 2016, available online: http://dx.doi.org/10.1080/01621459.2016.1260469

Keywords

high-dimensional regression, LASSO, penalized likelihood, posterior concentration, spike-and-slab, variable selection

Share

COinS
 

Date Posted: 25 October 2018

This document has been peer reviewed.