Date of this Version
Advances in Neural Information Processing Systems
We provide a natural gradient method that represents the steepest descent direction based on the underlying structure of the parameter space. Although gradient methods cannot make large changes in the values of the parameters, we show that the natural gradient is moving toward choosing a greedy optimal action rather than just a better action. These greedy optimal actions are those that would be chosen under one improvement step of policy iteration with approximate, compatible value functions, as defined by Sutton et al. . We then show drastic performance improvements in simple MDPs and in the more challenging MDP of Tetris.
Kakade, S. M. (2001). A Natural Policy Gradient. Advances in Neural Information Processing Systems, 14 Retrieved from https://repository.upenn.edu/statistics_papers/471
Date Posted: 27 November 2017