Date of this Version
Mathematics of Operations Research
We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin (1956) for temporally non-homogeneous Markov chains, and the principal innovation is that here the summands are permitted to depend on both the current state and a bounded number of future states of the chain. We show through several examples that this added flexibility gives one a direct path to asymptotic normality of the optimal total reward of finite horizon Markov decision problems. The same examples also explain why such results are not easily obtained by alternative Markovian techniques such as enlargement of the state space.
non-homogeneous Markov chain, central limit theorem, Markov decision problem, sequential decision, dynamic inventory management, alternating subsequence
Arlotto, A., & Steele, J. M. (2016). A Central Limit Theorem for Temporally Non-Homogenous Markov Chains with Applications to Dynamic Programming. Mathematics of Operations Research, 41 (4), 1448-1468. http://dx.doi.org/10.1287/moor.2016.0784
Date Posted: 27 November 2017
This document has been peer reviewed.