Date of this Version
Advances in Neural Information Processing Systems
We propose a fast algorithm for ridge regression when the number of features is much larger than the number of observations (p≫n). The standard way to solve ridge regression in this setting works in the dual space and gives a running time of O(n2p). Our algorithm Subsampled Randomized Hadamard Transform - Dual Ridge Regression (SRHT-DRR) runs in time O(np log(n)) and works by preconditioning the design matrix by a Randomized Walsh-Hadamard Transform with a subsequent subsampling of features. We provide risk bounds for our SRHT-DRR algorithm in the fixed design setting and show experimental results on synthetic and real datasets.
Lu, Y., Dhillon, P. S., Foster, D. P., & Ungar, L. H. (2013). Faster Ridge Regression via the Subsampled Randomized Hadamard Transform. Advances in Neural Information Processing Systems, 26 Retrieved from https://repository.upenn.edu/statistics_papers/221
Date Posted: 27 November 2017