TOPICS IN STATISTICAL MACHINE LEARNING

Loading...
Thumbnail Image
Degree type
Doctor of Philosophy (PhD)
Graduate group
Statistics
Discipline
Statistics and Probability
Computer Sciences
Mathematics
Subject
Bandits
Causal Inference
Machine Learning
Minimax Theory
Transfer Learning
Funder
Grant number
License
Copyright date
2022
Distributor
Related resources
Author
Pu, Hongming
Contributor
Abstract

Modern statistical machine learning combines statistics with the computational sciencesincluding computer science and optimization. The research in statistical machine learning facilitates the development of fields such as medicine, signal processing, bioinformatics, artificial intelligence and operation research and these fields provide practical problems that motivate statistical machine learning study. Nowadays, more and more opportunities can be found in statistical machine learning driven by these applied problems. This thesis aims to address the following three problems in statistical machine learning:

  1. In the first part, we study stochastic continuum-armed bandits with additive models. A near optimal algorithm is proposed and the minimax rate of regret is established. The results show an interesting phenomenon: the optimal regret is independent of the dimension if sparsity assumption is made, which highlights the difference between high-dimensional bandits and high-dimensional estimation. We also study the adaptivity issue of this problem and show it is possible to adapt to the sparsity but impossible to adapt to the smoothness. We then develop a new algorithm that can achieve near optimal regret adaptively under an additional assumption.
  2. In the second part, we study the problem of transfer learning for nonparametric regression. A near optimal algorithm is also established and the minimax optimal rate of risk is identified. We further propose a data-driven algorithm and show it simultaneously attains the optimal rate over a large collection of parameter spaces. Finally we extend this problem to the case where multiple source domains are considered.
  3. In the third part of this thesis, we study the optimal treatment rules with an instrumental variable in causal inference. We first propose a general framework for estimation of optimal individualized treatment rules with a valid intrumental variable. Under this framework, we then define a novel notion of optimality called IV-optimality for treatment rules. Finally we propose an estimator of an IV-optimal rule and prove theoretical guarantees.
Advisor
Cai, Tianwen
Date of degree
2022
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Volume number
Issue number
Publisher
Publisher DOI
Journal Issue
Comments
Recommended citation