Modern statistical machine learning combines statistics with the computational sciencesincluding computer science and optimization. The research in statistical machine learning facilitates the development of fields such as medicine, signal processing, bioinformatics, artificial intelligence and operation research and these fields provide practical problems that motivate statistical machine learning study. Nowadays, more and more opportunities can be found in statistical machine learning driven by these applied problems. This thesis aims to address the following three problems in statistical machine learning:

1. In the first part, we study stochastic continuum-armed bandits with additive models. A near optimal algorithm is proposed and the minimax rate of regret is established. The results show an interesting phenomenon: the optimal regret is independent of the dimension if sparsity assumption is made, which highlights the difference between high-dimensional bandits and high-dimensional estimation. We also study the adaptivity issue of this problem and show it is possible to adapt to the sparsity but impossible to adapt to the smoothness. We then develop a new algorithm that can achieve near optimal regret adaptively under an additional assumption.
2. In the second part, we study the problem of transfer learning for nonparametric regression. A near optimal algorithm is also established and the minimax optimal rate of risk is identified. We further propose a data-driven algorithm and show it simultaneously attains the optimal rate over a large collection of parameter spaces. Finally we extend this problem to the case where multiple source domains are considered.
3. In the third part of this thesis, we study the optimal treatment rules with an instrumental variable in causal inference. We first propose a general framework for estimation of optimal individualized treatment rules with a valid intrumental variable. Under this framework, we then define a novel notion of optimality called IV-optimality for treatment rules. Finally we propose an estimator of an IV-optimal rule and prove theoretical guarantees.