Road markers classification using binary scanning and slope contours

Road markers guide the driver while driving on the road to control the traffic for the safety of the road users. With the booming autonomous car technology, the road markers classification is important in its vision segment to navigate the autonomous car. A new method is proposed in this paper to classify five types of road markers namely dashed, single, double, solid-dashed and dashed-solid which are commonly found on the two lane single carriageway. The classification is using unique feature acquired from the binary image by scanning on each of the images to calculate the frequency of binary transition. Another feature which is the slopes between the two centroids which allow the proposed method, to perform the classification within the same video frame period. This proposed method has been observed to achieve an accuracy value of at least 93%, which is higher than the accuracy value achieved by the existing methods.


Introduction
The road markers are essential to control the road traffic and avoid traffic hazards by guiding the road users to safely decide either to maintain the course in the middle of the lane or to overtake the front vehicles. Road markers are categorized in different types subject to the types of the roads such as urban, non-urban and highway, on which the sizes of the road markers may slightly vary. Existing research work have been extensively carried out to solve the lane detection and tracking problems for Auto-Assist Driving System (ADS) [1][2][3][4][5][6]. The schemes proposed are mainly about detecting and tracking the lanes, without focusing on classifying the road markers. Others, had worked on the road symbols, alphabets and customize markers [7][8][9][10] which also used as the information to alert the drivers. Nevertheless, road marker classification still have some room for improvement as there are different types of markers can be found globally [11,12].
A frequency analysis method for the urban road marker classification had been proposed in [13] which uses the inverse perspective mapping (IPM) to produce a bird-eye view of the road and produce a modified Hough Transform (HT) for better lane detection. Then the power spectrum and Fourier analysis are applied to detect three types of markers, namely; single solid, dashed, and merged road markers which are commonly found at the urban road. Schubert et al. reported in [14] about a lane changing approach based on the road traffic surrounding situation including the type of the road marker. However the marker type's classification is limited to only dashed and solid road markers. In a work presented by Lindner et al. [15], an edge detector technique was designed to subsequently search for a different objects on the road to detect the road markers. Even though four types of road markerscan be detected, it only concentrates on the dashed lines with different sizes.
In another approach by Suchitra et al. [16], a modular approach is used to classify three road markers namely; dashed, solid and zigzag. To classify either dashed or solid, Basic Lane Marking (BLM), which is based on continuity properties is applied. This approach however, applies the temporal information in the classification operation, which renders slower detection when the road marker type changes whilst driving on the road. Nedevschi et al. [17] able to  [18][19][20] had worked on the five types of road marker. A detection method based on Bayesian approach for classifying the road markers has been proposed. Although it is able to classify five types of road markers, the method suffers outlier problems especially at transitions between two types of markers. The results of the classification were found to have changed abruptly on each frame, which caused inconsistent results while driving. Mathibela et al. [21] concentrate on the road markers found at the urban area and able to classify seven types, namely; double boundary, single boundary, zig-zag, separator, boxed junction, intersectionand special lanes by using a unique set of geometric features which function within a probabilistic RUSBoost and Conditional Random Field (CRF) network. However, this research is mainly carried out on a static image. The latest publication from Zamani et al. [22,23] using the features extracted from the road lane, and fed into the Artificial Neural Network (ANN). The initial research starts with two types of lane markers and later expanded to five types of markers classification [24] which the features extracted from the customized Region of Interest (ROI). Nevertheless, the algorithm requires temporal values from to lot of image sequence before it can make a decision on type's transition due to its small ROI. This may lags the information to alert the drivers. The aforementioned methods have been proposed to address different types of road markers subject to the physical features of the roads including the dimension, colour and size of the markers. In our research, the types of road markers which are normally found at the non-urban carriageway are classified using a machine vision system attached to a car. The road markers include solid, dashed, dashed-solid, double-solid and solid-dashed, as shown in Figure 1.

Research Method
The proposed classification process is using a two-layer classifier as shown as in Figure 2. First layer will classify three classes of markers whereas the next layer will classify another two more classes. The first layer is using the scanning the binary image to get the percentage of the binary transition and the second layer classifier is using the slope value from the two contours detected in the same image. Before all these process can be done, the Region of Interest (ROI) will be determined from the video image, and this would require the camera location to be adjusted at the front of the camera.

Camera Setup for Region of Interest (ROI) selection
A new method in selecting the ROI based on its calibrated camera's height and Field of View (FOV) are used. For the initial setup, a camera with a resolution of 1280x720 located at the center of the lane, is calibrated on its position with its FOV adjusted towards the planar road surface captured by the camera as shown in Figures 3 (a) and 3 (b). To calibrate the position of the camera, two horizontal lines, h1 and h2, are set on the FOV to equally divide into 3 sections as in Figure 3 (c), where the h1 horizontal line in the image must cross the intersection point between the rightmost lane, the off-road region and the border of the image to maximize the ROI.
The ROI selection is then applied to the video and the output for the ROI is labelled as r(x,y) as in Figure 4 which contains the information of the marker, by dividing the image into two halves before selecting two third of from the right, as indicated by the red box, which is bordered by the two horizontal lines. With this method, the ROI contains the largest visible road marker of the image.  The ROI, r(x,y) which is in two dimension (2D) will represents the video images in pixels and will go through the pre-processing prior to the feature extraction and classification process. The raw image in Red Green Blue (RGB) video images will be filtered through the Gaussian filter for 2D as in (1). The parameter x is the horizontal length from the origin and y is the vertical length from the origin and σ is the standard deviation of the Gaussian distribution [25].
This Gaussian operation will reduce the noise generated in the segmentation process to produce a good binary image for thresholding operation on Hue, Saturation and Luminance format (HSL). In this process, the normal RGB colour image is converted to HSL. Since the typically white lane marker is on a grey background, the luminance value is adjusted as the threshold, use to convert the image to binary format.

Extracting The Road Marking Cues
After investigating the best ROI for processing the image, the next step is to extract the road marking cues. The ROI will be scanned from the top to the bottom and from the left to the right as shown by the arrows in Figure 5 (b). A binary column vector , which is extracted from the x-th column of the binary image having binary pixels of ( , ) ∈ {0,1} for = 0,1, ⋯ , and = 0,1, ⋯ , , is defined as follows This binary column vector tells how many white pixels found on the x-th column. When there are road markers crossing the column, the number of white pixels will increase. In order to reduce the computational complexity, not all columns will be read, or horizontally scanned over the image. It is sufficient to horizontally scan the image for obtaining the values over a constant horizontal interval of δx as in Figure 5 (b), which is the number of pixel columns of the binary image which are skipped and not scanned, as long as the road marker can be correctly classify. Example of scanning process on the image shown in Figure 5, with various type of road markers and different value of δx.
By using an example of a 420x260 pixel size for an ROI, it will requires 41 times of lines scanning using with interval size of δx=10 pixels, producing 41 binary column vectors , where = 260 (the Height size). Every binary column vector, will be processed to detect and count the number of transitions, from 0 to 1 and from 1 to 0 in it by using a bitwise exclusive OR operation between itself, and , which is a vector produced from shifting by one binary bit or element as shown in an example. The numbers of transitions in each binary column vector obtained from the scanning line are saved into a vector, . An example of this vector produced from scanning an ROI with an interval size of 10 pixels is given in Figure 6. The number of transitions are in each binary column vector is observed to be either 0, 2 or 4. For the road marker SS, there are at most 2 transitions in each vector whereas for D, the number of transitions is either 0 or 2. For DD, the number of transitions is 4, whereas for SD and DS, the number of transitions is either 2 or 4. To differentiate between SD and DS, additional features will be extracted to differentiate between these two classes. Vector will be processed where the number of transitions will be grouped and the percentage of , or the numbers of transitions will be stored in vector as formulated by p in (3), which is applied to calculate the percentage of for five different classes as tabulated in Table 1

The First and Second Layer Classification 2.3.1. Designing the Filtering Table for First Layer Classification
The percentages of obtained when processing 100 frames from each of the five classes are plotted in Figure 7. The X-axis represents whereas Y-axis represents p , which is the percentage of . In our method, we use 10 pixels as the interval size for the scanning process. From observation, a threshold can be set from the data to enable the classification for the markers. At threshold value of 20% as shown as red dash line, a filtering table can be designed as in Table 2. At least 2 types of markers can be classified, whereas SS and DS/SD classes could produced the same result. To improve this problem, an additional percentage value is introduced, which is 4 p = 6, which is a multiplication of 2 p = 2 and 3 p = 4 and later gain by 2 as formulated in (4) are plotted in Figures 7 (a)-(e) as inside the blue dash box. From this additional percentage value, the filtering table can be improved as in Table 2 to classify three road marker types namely D, DD and SS. As for SD and DS road marker type, the algorithm will proceed to the second classifier as described in the next section.

Contours Slope (Second Layer Classifier)
In the second layer classification, the aim is to separate the DS and SD classes which requires the process to detect the contours (long and short markers). A DS class is recognized by the location of the shorter contour (of the dashed marker) below the longer contour (the solid marker) and an SD class is recognized vice versa. In Figure 8, the longer contour is indicated by two points, A and B, and its both ends. The shorter contour is indicated by point C, which is the centroid of the shorter contour. Hence by knowing the location of C, whether it's below or above the longer contour, the class SD or DS can be identified. , between A and C will be smaller than the slope of = − − between A and B, if the shorter contour is below the longer contour which is found in DS class. On the other hand, the slope of , will be larger than the slope, , if the shorter contour is below the longer contour for SD class. If both and are multiplied with ( )( ) , then the result of the multiplication, will still be smaller than the result of another multiplication for DS class.
If the difference between these two multiplications is denoted as  , the following equation yields then the value of  can be used as the feature to determine or segregate between SD and DS classes. If the  < 0, then the road marker is classified as DS class. If , 0   then the road marker is classified as SD class. Since the classification is only carried out to identify between

2055
SD and DS for this second layer classification, the value of  will not equivalent as zero as C will never be located on the straight line between A and B.

Outliers and Improving the Classification Result
By observing the graphs in Figure 7, there are potential outliers generated due to the values calculated near to the threshold values at 20% as indicated by the red circle in Figures 7 (a), (c) and (e). Other than introducing 4 p = 6, this possible false error can also be resolved by using temporal values between the video images. The markers tend to be the similarly labeled for a set of adjacent frames. By using this possibility, the previous classification result would be stored and if there is a transition of a class or marker type, it will confirm on the next ten (10) frames of sequence. This method also improved the issue related to the vehicle blocking the road marker as there is no firm classification result when this occurs and the result will hold the last classification result as reference. Hence the validation for transition types of marker requires 10*0.033s=0.33s at the most.

Results and Analysis
The proposed two-layer classification methods are implemented to classify the road markers from a number of video clips recorded when driving at different road scenarios with different road markers. The speed of the car while these video clips were taken was between 70-80 km/hr. In order to run the proposed road marker classification model, the features from the first three video clips given in Table 3 are extracted and analyzed to see the significant patterns for the classification process and it contain all types of the road markers which are aimed to be classified. Manual comparison is performed on the predicted road marker type of the algorithm with the ground truth on every frame and the accuracy is calculated as in (8) where the total frames classified correctly from the ground truth ∑ is divided with the total frames, ∑ . It can be observed that the accuracy achieved in the training videos are well above 98% except clip number 3 at 93% which due to the faint lane markers detected in the video. This is essential to ensure that the classification performance achieved when running this model on the testing videos is as good as possible.  The proposed classification model was later tested with five video clips, which are numbered from 4 to 8 in Table 3. In video clip Vid_2_Test, which contains 796 frames having road markers of types DD and D, the recorded accuracy is 94.47%, which still higher than 90% but stands as the lowest amongst all other accuracy values achieved. This is mainly due to the quality of this video clip which is more blurry compared on the other clips. The marker transition had shown minimum effect on the classification accuracy as the algorithm successfully revalidated between the false and true transition and contributed on the accucarcy calculation.
As for the confusion matrix as in Table 4, we can find out that most error is coming from the DD class due to the thresholding process resulting the output of combined contour at the top of the lanes which is very hard to distinguish its threshold value. Examples of the thresholded images are as in Figure 9 as in the blue circle. In this circle area, the scanning process will resulted this as = 2 instead of = 2 and the classification would lead to SD instead of DD.

Conclusion
The paper presented a new approach for lane marker classification. The ROI selection is based on the camera position to capture the road image. The ROI is processed and the features of the lane marking are extracted for the training purposes and later for the classification process. This new algorithm was tested with the local new videos and the ground truth. Experimental results have shown that the accuracy values achieved by the proposed new method achieved more than 93%. The errors from the classification operation are mainly due to the faint road marker which caused wrong pre-processing result, sudden illumination changes and the difference in the dimension of the lane markers.