Abstract: A new integrating planning SARSA (λ) algorithm of reinforcement learning is proposed. The algorithm makes extremely efficient use of the experience data. It learns the model while learning to estimate the optimal value without a model. After each episode, it plans in the table of state/action pairs being recorded, and the table can be as a quantificational trade off reference between learning and planning. The results of experiment show that the algorithm has better performance than the SARSA (λ) algorithm.