Abstract: In order to deal with the drawbacks of template drifting in small sample space and the bad real-time performance in large sample space with the tracking algorithm based on sparse representation model, a tracking approach based on double sparse representation and circle shape sampling was proposed. The image data obtained from tracking rectangular frame were sampled with circle shape sampling model, not only preserving grayscale and structure information of tracking object, but also cutting down the disturbance from background pixels. Meanwhile, the trivial template coefficients gotten from sparse representation were analyzed with a distance weighting function to be used for obtaining target sample changing condition and improving efficiency of template updating. Finally, HOG(histogram of oriented gradient)feature was introduced for once more sparse representation to the second-best sparse solutions, which can cut down estimation error in small sample space. Experimental results show that the proposed algorithm can improve the robustness and efficiency of object tracking in small sample space.