SLOPE is Adaptive to Unknown Sparsity and Asymptotically Minimax

Loading...
Thumbnail Image
Penn collection
Statistics Papers
Degree type
Discipline
Subject
SLOPE
Lasso
sparse regression
adaptivity
false discovery rate (FDR)
Benjamini-Hochberg procedure
FDR thresholding
Physical Sciences and Mathematics
Funder
Grant number
License
Copyright date
Distributor
Related resources
Author
Su, Weijie
Candès, Emmanuel
Contributor
Abstract

We consider high-dimensional sparse regression problems in which we observe y = Xβ + z, where X is an n × p design matrix and z is an n-dimensional vector of independent Gaussian errors, each with variance σ2. Our focus is on the recently introduced SLOPE estimator [Ann. Appl. Stat. 9 (2015) 1103–1140], which regularizes the least-squares estimates with the rank-dependent penalty ∑1≤i≤pλi|βˆ|(i), where |βˆ|(i) is the ith largest magnitude of the fitted coefficients. Under Gaussian designs, where the entries of X are i.i.d. N(0,1/n), we show that SLOPE, with weights λi just about equal to σ⋅Φ−1(1−iq/(2p)) (Φ−1(α) is the αth quantile of a standard normal and q is a fixed number in (0,1)) achieves a squared error of estimation obeying sup‖β‖0≤kℙ(‖βˆSLOPE−β‖2>(1+ε)2σ2k log(p/k)) ⟶ 0 as the dimension p increases to ∞, and where ε>0 is an arbitrary small constant. This holds under a weak assumption on the sparsity level k and is sharp in the sense that this is the best possible error any estimator can achieve. A remarkable feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of sparsity classes. We are not aware of any other estimator with this property.

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2016-01-01
Journal title
The Annals of Statistics
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation
Collection