Statistics Papers

Document Type

Conference Paper

Date of this Version

2-12-2014

Publication Source

Journal of Machine Learning Research

Start Page

1

Last Page

27

Abstract

We establish optimal rates for online regression for arbitrary classes of regression functions in terms of the sequential entropy introduced in [14]. The optimal rates are shown to exhibit a phase transition analogous to the i.i.d./statistical learning case, studied in [16]. In the frequently encountered situation when sequential entropy and i.i.d. empirical entropy match, our results point to the interesting phenomenon that the rates for statistical learning with squared loss and online nonparametric regression are the same. In addition to a non-algorithmic study of minimax regret, we exhibit a generic forecaster that enjoys the established optimal rates. We also provide a recipe for designing online regression algorithms that can be computationally efficient. We illustrate the techniques by deriving existing and new forecasters for the case of finite experts and for online linear regression.

Copyright/Permission Statement

This paper is published under a Creative Commons copyright license to the general public, in particular a Creative Commons Attribution 4.0 International License, which is incorporated herein by reference and is further specified at http://creativecommons.org/licenses/by/4.0/legalcode (human readable summary at http://creativecommons.org/licenses/by/4.0).

Share

COinS
 

Date Posted: 27 November 2017

This document has been peer reviewed.