一种快速强化学习方法研究
Research on Fast Reinforcement Learning
-
摘要: 在对资格迹理论研究的基础上,提出了一种延迟快速强化学习算法DFSARSA(λ)(延迟快速SARSA(λ)算法).算法的主要思想是通过对资格迹的重新定义和对即时差分TD(λ)偏差的跟踪,使强化学习中Q值在需要时进行更新,而SARSA(λ)每一步都对Q值进行更新,该方法使SARSA(λ)算法的更新计算复杂度从O(|S||A|)降到了O(|A|),提高了强化学习速度,仿真实验证明了该算法的有效性.Abstract: Based on eligibility trace theory, a delayed fast reinforcement learning algorithm DFSARSA(λ) is proposed in this paper. By redefining the eligibility trace and tracking the (TD(λ)) error, the Q-value of reinforcement learning updates may be postponed when they are needed instead of update in each step as traditional SARSA(λ). The update computing complexity is reduced from O(|S||A|) to O(|A|) compared with SARSA(λ) and the speed of the reinforcement learning is improved greatly. Simulation results show the method's validity.
下载: