Abstract: In order to improve the performance of highway lane departure warning system, a lane detection and tracking system was presented based on region dynamically planning and dual model fitting. The image pre-processing was featured by a method of template erosion and dilation to remove the interference information. According to the infinitesimal theory, the picture was divided into small pieces, which were extracted separately, and analyzed as a whole. Finally the straight line and B-spline models were applied to simulate the lane. Test results of this system show that the system and its arithmetic improve the adaptability to different lane markings.