Statistics Papers

Document Type

Journal Article

Date of this Version

11-2016

Publication Source

Mathematics of Operations Research

Volume

41

Issue

4

Start Page

1448

Last Page

1468

DOI

10.1287/moor.2016.0784

Abstract

We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin (1956) for temporally non-homogeneous Markov chains, and the principal innovation is that here the summands are permitted to depend on both the current state and a bounded number of future states of the chain. We show through several examples that this added flexibility gives one a direct path to asymptotic normality of the optimal total reward of finite horizon Markov decision problems. The same examples also explain why such results are not easily obtained by alternative Markovian techniques such as enlargement of the state space.

Keywords

non-homogeneous Markov chain, central limit theorem, Markov decision problem, sequential decision, dynamic inventory management, alternating subsequence

Share

COinS
 

Date Posted: 27 November 2017

This document has been peer reviewed.