A highway visual tracking system using UAV based digital image. This highway tracking system created by using computer vision. The methods used in this system are the RGB - HSV conversion, colour detection, edge detection, thresholding, dilation canny edge detection, Hough transform, and moments. Prior to image processing, capture video of the highway. In image processing, highways video is prepared to enter image preprocessing, then administer the thresholding and dilation process. This was followed by the method of moments to get the coordinates of the first phase tracking. Next, the X-coordinate is used as a reference to determine the region of interest (ROI). ROI is processed using the Canny edge detection followed hough transform to refine the detected lines into the desired line. Coordinates x line average can be computed which then shows the position of the highway. The programs used in the process of road tracking are using OpenCV 2.3.1 and Visual Studio 2010. The programming language used is C++. Tests carried out using video of highway taken directly from Youtube. The highway HSV colour values are in the range of 20, 3, 30-180, 32.169 and the environment around the highway consists of green vegetation, terrain, office buildings, houses, and trade stalls. The materials that affect the outcome of visual tracking are the presence of a vehicle on the highway and other objects that exist around the highway.