Detection of lane line based on Robert operator

2.1. Image edge detection operator

The essence of edge detection is to use algorithms to extract the boundary between the object and the background in the image. The edge is defined as the boundary of the region where the gray scale changes sharply. The classic edge detection method is to construct an edge detection operator based on a small neighborhood of pixels in the original image. The following is a theoretical analysis of several classic edge detection operators.

(1) First-order edge detection operator.

In an image, if the gray value change between adjacent points is small, the amplitude of the gradient will be small, and on the contrary, the amplitude of the gradient will also be large. Therefore, the edge pixel position of the image can be obtained by calculating the derivative. In the actual image during processing, often using a small convolution template to replace the partial derivatives of by-pixel operation. According to the structure of the template, a variety of first-order edge detection operators can be constructed.

The commonly used first-order operators are as follows:

Robert operator is also called cross differential algorithm. It is a gradient algorithm based on cross difference. It detects edge lines through local difference calculation. It is commonly used to process steep and low-noise images. When the edge of the image is close to plus or minus 45 degrees, the processing effect of this algorithm is more ideal. The disadvantage is that the edges of the extracted image are rough.

The template of Robert operator is divided into horizontal direction and vertical direction, as shown in Eq. (1). It can be seen from the template that the Robert operator can better enhance the edge of the image at plus or minus 45 degrees. Where $d_x$ represents the gradient in the $X$ direction and $d_y$ represents the gradient in the $Y$ direction:

Prewitt operator is a differential operator for image edge detection. Its principle is to use the difference generated by pixel gray value in a specific area to realize edge detection. Because the Prewitt operator uses the 3×3 template to calculate the pixel values in the area, and the Robert operator uses the 2×2 template. Therefore, the edge detection result of Prewitt operator is more obvious in both horizontal and vertical directions than Robert operator. The Prewitt operator is suitable for identifying images with a lot of noise and gradual gray scale. The calculation is shown in Eq. (2): where $d_x$ represents the gradient in the $X$ direction and $d_y$ represents the gradient in the $Y$ direction:

Sobel operator is a discrete differential operator for edge detection, which combines Gaussian smoothing and differential derivation. This operator is used to calculate the approximate value of the brightness of the image. According to the brightness of the edge of the image, the specific points in the area exceeding a certain number are recorded as edges. The Sobel operator adds the concept of weight on the basis of the Prewitt operator. It is believed that the distance between adjacent points has different effects on the current pixel. The closer the pixel point corresponds to the greater the impact of the current pixel, so as to achieve image sharpening and highlight the edge contour.

The Sobel operator detects the edge by reaching the extreme value at the edge according to the weighted difference of the gray scale of the upper and lower pixels and the left and right adjacent points. It has a smoothing effect on noise and provides more accurate edge direction information. Because the Sobel operator combines Gaussian smoothing and differential derivation, the result will have more noise resistance. When the accuracy requirements are not very high, Sobel operator is a more commonly used edge detection method.

Sobel edge operator more accurate positioning sub, commonly used in much noise, shades of gray image. The algorithm template is shown in Eq. (3), where $d_x$ represents the horizontal direction and $d_y$ represents the vertical direction:

(2) Second-order edge detection operator.

The second-order edge detection operator uses the feature that the second-order derivative at the edge of the object in the image is zero for edge detection. For the edge of the object in the image, the pixel value at the edge will undergo a step change, and the signs of the second derivative on both sides of the edge point are opposite.

Commonly used second-order operators are as follows:

The Canny algorithm is the best solution between suppressing noise and accurately retaining edge features [12, 13]. Canny edge detection is actually a first-order differential operator detection algorithm, which uses the step property of the image at the edge, that is, the characteristic that the image gradient obtains a maximum value at the edge for edge detection. Gradient is a vector, it has direction and modulus:

The direction of the gradient provides edge trend information, because the gradient direction is always perpendicular to the edge direction, and the magnitude of the gradient modulus provides edge strength information. In actual use, finite difference is usually used for gradient approximation. For the above formula, there are the following approximations:

2.2. Comparative analysis of four edge detection operators

Using MATLAB R2019b on the Windows 10 operating system with a CPU of 1.60 GHz and RAM of 8 GB, four operator edge detection methods are used to perform edge detection processing on the same lane image, as shown in Fig. 1. It can be clearly seen from the figure that the edges of the lane markings and the protruding edges are outlined by white contour lines, and the non-contoured parts are presented in black. In the first-order operator edge detection, the Prewitt operator detects that the image has more white points outside the white contour line, and the detection accuracy is poor. In the images detected by the Sobel operator and the Robert operator, the white points are similar, so in the first-order operator detection edge detection, the Sobel operator and the Robert operator have relatively high edge detection accuracy. In the second-order edge detection, in the Canny operator edge detection method, the image has no messy white spots, and the image to be detected has a clear outline, which is much better than the first-order edge detection method. Therefore, among the edge detection results of the four edge detection operators, the Canny operator edge detection result is more accurate.

The lane edges are detected by 4 methods. The 4 detection methods are run 100 times respectively, and the average value is taken. After statistics, the average value of the detection time of the edge detection operator is obtained, as shown in Table 1.

According to the data in the table, among the four edge detection operators, the Robert operator edge detection takes the least time, only 0.037972 s, while the traditional Canny operator edge detection method takes the longest, and the detection process uses 0.047919 s. By comparison, in terms of detection speed, Robert operator is 0.009947 s faster than Canny operator. Robert operator edge detection takes the least time in lane edge detection, and is more suitable for rapid detection of lane lines.

Through the comparative and comprehensive analysis of the four edge detection operators in image detection and detection time, Canny operator is better than Robert operator in image detection sharpness, but the white dots appearing on Robert operator are for the lane The detection of the line has less impact and does not affect the detection of the final lane line. In the comparative analysis of the time used for edge detection, it can be seen that the Robert operator takes a shorter time in image detection than the Canny operator, which only requires 0.037972 s. Based on the pursuit of detection timeliness, this paper chooses Robert operator as the edge detection method.

Fig. 1Image edge detection and processing

a) Robert algorithm

b) Sobel algorithm

c) Prewitt algorithm

d) Canny algorithm

3.1. Lane line detection process

This article is based on the Hough transform lane line extraction. The specific detection process is shown in Fig. 2.

Fig. 2Lane detection process

1) Original image acquisition.

During the driving of the car, the camera converts the captured optical image into a digital signal, which is sent to the image capture card for digital processing. This article uses the Sony HDR-XR520E monocular high-definition camera to collect the original lane line map. Install the camera in the middle of the front windshield and adjust the shooting angle of the lens until the lane line is in the middle of the image. The original image collected is shown in Fig. 3(a).

2) Image gray-scale processing.

The images collected by the vision sensor are mostly color images. Since the three-channel model of color image contains a lot of information that has nothing to do with the detection target, it is time-consuming to process and the real-time performance will be reduced. Therefore, this article converts the collected road color image into a grayscale image, which is equivalent to subtracting two-thirds of the original data, and the lane line description information is retained in the grayscale image, which can meet the needs of lane line detection. Most color images now use the RGB color mode. When processing the image, the three components of R (red), G (green), and B (blue) must be processed separately. Herein gradation processing using the image component method:

After obtaining the original image, in order to facilitate the identification, the image gradation processing, in MATLAB, a gradation processing function pic_gray = rgb2gray (ori), after gradation processing shown in Fig. 3(b). After the image is grayscaled, the color part of the image is replaced by gray and white, which makes the outline of the image clear and easy to recognize.

Fig. 3Preprocessing of image edge detection

a)

b)

c)

d)

3) Image filtering.

Image filtering, that is, suppressing the noise of the target image while preserving the details of the image as much as possible, is an indispensable operation in image preprocessing, and its processing effect directly affects the effectiveness and reliability of subsequent image analysis. Median filtering is based on the value of the central point position of this sequence is replaced with the median value of the point field. Its advantages are simple operation, fast speed, and good denoising effect.

Therefore, this paper uses the median filter method to filter the image. In MATLAB, the filter processing function is filter1_pic=medlt2(pic_grafiy), and the image after filtering is shown in Fig. 3(c). After filtering the image, the noise of the image is reduced, the contrast of the image is enhanced, and the image is easily recognized.

4) Image binarization.

Image binarization is the process of setting the gray value of the pixels on the image to 0 or 255, that is, the entire image presents an obvious black and white effect. In digital image processing, binary image occupies a very important position, binary image of the image in order to reduce the amount of data, which can highlight the contour of an object.

In MATLAB, the image binarization function is bw_pic=im2bw(filter2 pic). The binarized image is shown in Fig. 3(d). The outline of the road line of the image is clearer and becomes white, while the non-road markings are made black.

5) Image boundary detection.

This paper uses Robert operator and Canny operator to detect the edge of the lane line respectively. In MATLAB, edge(bw pic,'Robert') means using Robert operator to extract edge information, as shown in Fig. 4(a). edge(bw pic,'Canny') means using Canny operator to extract edge information, as shown in Fig. 4(b). It can be seen from the figure that the outline of the lane line becomes clearer, the overall lane line outline is more distinct, the lane edge image has a white outline wrapped, and the other non-lane boundary lines are black areas. There is little difference between the detection of the two operators.

Fig. 4Image edge detection

a)

b)

6) Hoff transform.

Hough transform is a feature extraction, which is widely used in image analysis, computer vision and digital image processing. Hough transform is used to identify features in objects. The algorithm flow is roughly as follows. Through the rough outline of the image and the recognition of the graphics, the algorithm will perform voting in the parameter space to determine the shape of the object, and this is determined by the local maximum in the accumulation space.

In MATLAB, the function for Hough transform processing is [H,T,R]=hough(verge_pic), H is the X-axis data, T is the Y-axis data, and R is the initial magnification value of the image. The image through the Hough transform is shown in Fig. 5(a) and 5(b) are the Hough transform after edge detection of Roberts operator and Canny operator. It can be seen from the figure that only the $Y$ coordinate value of the right point of the two pictures differs by 1, and the other values are all Unanimous.

7) Drawing and marking of lane lines.

After the Hough transform, the lane lines are drawn on the image. In MATLAB, use the function n=polyval(p,t) to draw the lane line as shown in Fig. 6(a) and 6(b) are the images after the integration of Roberts operator and Canny operator, respectively. It can be seen from the images that both images can clearly show the extracted lane lines. Based on the lane line drawing, the detected lane line is then marked on the original image, as shown in Fig. 7. The green detection line is accurately marked on the original lane line.

Fig. 5Hough transform

a)

b)

Fig. 6Lane line drawing

a)

b)

Fig. 7Lane line labeling

a)

b)

After comparison, it is found that the image detected by the Robert operator and the detected image processed by the Canny operator have little difference between the Hough transform, lane line extraction and lane line labeling, and both can clearly show the lane. The position of the line. Therefore, the integration of the Robert operator is the same as the traditional Canny operator, which can realize the detection of lane markings.

4.1. Comparison of the similarity of the test images

Based on gray histogram matching, the color image is divided into three channels (RGB), and the respective histograms are calculated, and the histogram matching is performed on the RGB three channels of the two images respectively, and finally the three matching results are averaged. The applied algorithm is as follows:

Import the two pictures a) and b) in Fig. 7 into the program in MATLAB, and the comparison results are shown in Fig. 8.

Fig. 8Comparison of similarity of test results

After comparison, it is found that the similarity of the lane line detection image after the edge processing of the Robert operator and the Canny operator is 0.99576. The results show that the Robert operator edge detection method is integrated into the Hough transform lane line detection method, and it has little effect on the final image clarity of the lane line detection.

4.2. Time comparison of test results

Using MATLAB R2019b on the Windows 10 operating system with a CPU of 1.60 GHz and RAM of 8GB, the Robert operator edge detection and Canny operator edge detection methods are integrated into the Hough transform lane line detection method to detect the lane lines. Detection 100 run with the results shown in Table 2.

From the data in the table, we can see that the time used for each test fluctuates compared with the last time. The time used for 100 test runs is averaged, and the processing is shown in Table 3.

Through the comparison of the data in the table, it can be found that after the Robert operator and Canny operator are integrated into the Hough transform to detect the lane line, the Robert operator has a greater impact on the detection of the entire lane line than the Canny operator, and the entire detection time is reduced from 0.705261 s to 0.553351 s, the entire lane detection time is shortened by 0.15191 s. The integration of the Robert operator into the Hough transform lane line greatly shortens the time of lane line detection, thereby improving the timeliness of lane line detection.