Date of this Version
Journal of Machine Learning Research
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a principal component analysis) and then performs an ordinary (un- regularized) least squares regression in this subspace. This note shows that the risk of this ordinary least squares method (PCA-OLS) is within a constant factor (namely 4) of the risk of ridge regression (RR).
risk inflation, ridge regression, pca
Dhillon, P. S., Foster, D. P., Kakade, S. M., & Ungar, L. (2013). A Risk Comparison of Ordinary Least Squares vs Ridge Regression. Journal of Machine Learning Research, 14 1505-1511. Retrieved from https://repository.upenn.edu/statistics_papers/135
Date Posted: 27 November 2017
This document has been peer reviewed.