Abstract: The application of YOLO (you only look once) algorithm in gesture recognition was studied to improve the speed and accuracy of detection under the background near the skin color, light and shade. Based on the end-to-end detection function, the YOLO algorithm could be arranged to improve operation speed greatly by automatically extracting target feature from convolution neural networks. Considering the excellent performance in target detection process, YOLO algorithm was applied to gesture recognition. Comparing with other application results with YOLO series algorithm, this application result of YOLO algorithm shows better performance in gesture recognition. At the same time, based on a YOLOv3-tiny algorithm, the fast version of YOLOv3 algorithm, a YOLOv3-tiny-T algorithm was proposed. The YOLOv3-tiny-T algorithm can achieve a mean average precision of 92.24% on the UST dataset with five gestures, increasing about 5% combined with YOLOv3-tiny.