Abstract
This paper presents a reinforcement learning algorithm for solving innite horizon Markov Decision Processes under the expected total discounted reward criterion when both the state and action spaces are continuous. This algorithm is based onWatkins' Q-learning, but uses Nadaraya-Watson kernel smoothing to generalize knowledge to unvisited states. As expected, continuity conditions must be imposed on the mean rewards and transition probabilities. Using results from kernel regression theory, this algorithm is proven capable of producing a Q-value function estimate that is uniformly within an arbitrary tolerance of the true Q-value function with probability one. The algorithm is then applied to an example problem to empirically show convergence as well. © 2014 AI Access Foundation. All rights reserved.
Original language | American English |
---|---|
Pages (from-to) | 705-731 |
Number of pages | 27 |
Journal | Journal of Artificial Intelligence Research |
Volume | 49 |
DOIs | |
State | Published - Apr 29 2014 |
Keywords
- Kernel regression theory
- Kernel smoothing
- Markov decision processes
- Nadaraya--Watson
- Q-learning
- Q-value function estimate
- Reinforcement learning
- State and action spaces
DC Disciplines
- Education
- Mathematics