A Risk Comparison of Ordinary Least Squares vs Ridge Regression

Loading...
Thumbnail Image
Penn collection
Statistics Papers
Degree type
Discipline
Subject
risk inflation
ridge regression
pca
Computer Sciences
Funder
Grant number
License
Copyright date
Distributor
Related resources
Author
Dhillon, Paramveer S.
Foster, Dean P
Kakade, Sham M
Ungar, Lyle
Contributor
Abstract

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a principal component analysis) and then performs an ordinary (un- regularized) least squares regression in this subspace. This note shows that the risk of this ordinary least squares method (PCA-OLS) is within a constant factor (namely 4) of the risk of ridge regression (RR).

Advisor
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Publication date
2013-06-01
Journal title
Journal of Machine Learning Research
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation
Collection