Road Markings Condition Assessment Method for Intelligent Vehicles

doi:10.12141/j.issn.1000-565X.220126

Abstract

Abstract:

With the continuous advancement and popularization of autonomous driving technology, more and more vehicles with autonomous driving technology will appear on the road, and the service objects of road markings will gradually transition from drivers to autonomous vehicles.On the one hand, the method of road markings condition assessment requires a lot of manpower to inspect, measure and evaluate; on the other hand, the evaluation index is based on biological vision research, which does not conform to the characteristics of automatic driving vehicles based on machine vision. To solve the above problems, this paper proposed a method of road markings condition assessment for autonomous vehicles. First, PSNR(peak signal-to-noise ratio) was initially determined as the evaluation index by means of literature review, analogical reasoning and logical reasoning. Secondly, to quickly obtain PSNR, this paper proposes a calculation method of the PSNR based on image inpainting, which utilizes the DeblurGAN model restores the damaged road markings at the image level, and then uses the damaged and restored road markings images to calculate the PSNR. In addition, this paper proposed a data augmentation method that can realis-tically synthesize damaged road markings images to improve the performance of image inpainting models. Then, the AlexNet network was used as the benchmark model to design experiments to study the relationship between the PSNR and the recognition accuracy of road markings. The experimental results show that, compared with the calculation method of the PSNR based on the artificially restored image, the average PSNR obtained by the method proposed in this paper only differs by about 2.24%, but the acquisition speed is increased by about 418 times; when the average PSNR differs by about 43.66%, the average recognition accuracy differs by about 36.27%. Therefore, the PSNR can measure the use of road markings. Compared with the evaluation method of the current standard, the evaluation method proposed in this paper improves the work efficiency by about 6.5 times and consumes less manpower. And it is more in line with the characteristics of self-driving cars, but the evaluation methods are more detailed in the specification.

Key words: intelligent vehicles, road markings, working condition, assessment method, PSNR, recognition accuracy

CLC Number:

U495

FU Xinsha, PENG Jinhui, ZENG Yanjie, et al. Road Markings Condition Assessment Method for Intelligent Vehicles[J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(11): 1-13.

Figures/Tables 25

Fig.1

Fig.2

Fig. 3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Table 1

Fig.11

Fig.12

Fig.13

Fig.14

Table 2

Fig.15

Table 3

Table 4

Fig.16

Table 5

Fig.17

Fig.18

Table 6

Table 7

References 23

1	道路交通标志和标线：［S］.
2	道路交通标线质量要求和检测方法：［S］.
3	Manual on uniform traffic control devices ［Z］．Washington DC：Department of Transportation FHA，2009.
4	YE F， CARLSON P J， MILES J D ．Analysis of nighttime visibility of in-service pavement markings［J］．Transportation Research Record：Journal of the Transportation Research Board，2013，2384（1）：85-94.
5	XU S， WANG J， WU P，et al ．Vision-based pavement marking detection and condition assessment：a case study［J］．Applied Sciences，2021，11（7）：3152.
6	VOKHIDOV H， HONG H， KANG J，et al ．Recognition of damaged arrow-road markings by visible light camera sensor based on convolutional neural network［J］．Sensors，2016，16（12）：2160.
7	KAWANO M， MIKAMI K， YOKOYAMA S，et al ．Road marking blur detection with drive recorder［C］∥Proceedings of the 2017 IEEE International Conference on Big Data （IEEE Big Data）.Boston：IEEE，2017：4092-4097．
8	HIROYA M， YOSHIHIDE S， TOSHIKAZU S，et al ．Road damage detection and classification using deep neural networks with smartphone images［J］．Computer‐Aided Civil and Infrastructure Engineering，2018，33（12）：1127-1141.
9	雷蕴奇，柳秀霞，宋晓冰，等．视频中运动人脸的检测与特征定位方法［J］．华南理工大学学报（自然科学版），2009，37（5）：31-37.
	LEI Yun-qi， LIU Xiu-xia， SONG Xiao-bing，et al ．Face detection and feature location of moving men in video［J］．Journal of South China University of Technology （Natural Science Edition），2009，37（5）：31-37.
10	刘富，刘璐，侯涛，等．基于优化MSR的夜间道路图像增强方法［J］．吉林大学学报（工学版），2021，51（1）：323-330.
	LIU Fu， LIU Lu， HOU Tao，et al ．Night road image enhancement method based on optimized MSR［J］．Journal of Jilin University（Engineering and Technology Edition），2021，51（1）：323-330.
11	蒋渊德，孙朋朋，秦孔建，等．雨雪天气对自动驾驶视觉图像质量的影响［J］．中国公路学报，2022，35（3）：307-316.
	JIANG Yuan-de， SUN Peng-peng， QIN Kong-jian，et al ．Influence of rain and snow on autonomous vehicles’ camera［J］．China Journal of High-way and Transport，2022，35（3）：307-316.
12	孙季丰，朱雅婷，王恺．基于DeblurGAN和低秩分解的去运动模糊［J］．华南理工大学学报（自然科学版），2020，48（1）：32-50.
	SUN Jifeng， ZHU Yating， WANG Kai，et al ． Motion deblurring based on DeblurGAN and low rank decomposition［J］．Journal of South China University of Technology （Natural Science Edition），2020，48（1）：32-50.
13	IIZUKA S， SIMO-SERRA E， ISHIKAWA H ．Globally and locally consistent image completion［J］．ACM Transactions on Graphics，2017，36（4）：107/1-14.
14	QUAN W， ZHANG R， ZHANG Y，et al ．Image inpainting with local and global refinement［J］．IEEE Transactions on Image Processing，2022，31：2405-2420.
15	KUPYN O， BUDZAN V， MYKHAILYCH M，et al ．DeblurGAN：blind motion deblurring using conditional adversarial networks［C］∥Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）．Salt Lake City：IEEE，2018：8183-8192.
16	ISOLA P， ZHU J， ZHOU T，et al ．Image-to-image translation with conditional adversarial networks［C］∥Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition （CVPR 2017）．Hawaii：lEEE，2017：5967-5976．
17	GULRAJANI I， AHMED F， ARJOVSKY M，et al ．Improved training of wasserstein GANs［C］ ∥Proceedings of the 31st Advances in Neural Information Processing Systems （NIPS 2017）．California：NIPS，2017：5767-5777．
18	JOHNSON J， ALAHI A， FEI-FEI L ．Perceptual losses for real-time style transfer and super-resolution［C］∥Proceedings of the European Conference on Computer Vision．Amsterdam：Springer，2016：694-711．
19	KRIZHEVSKY A， SUTSKEVER I， HINTON G E ．ImageNet classification with deep convolutional neural networks［J］．Communications of the ACM．2017，60（6）：84-90.
20	LECUN Y， BOTTOU L， BENGIO Y，et al ．Gradient-based learning applied to document recognition［J］．Proceedings of the IEEE，1998，86（11）：2278-2324.
21	SIMONYAN K， ZISSERMAN A ．Very deep convolutional networks for large-scale image recognition［EB/OL］．arXiv preprint arXiv：.
22	SZEGEDY C， LIU W， JIA Y，et al ．Going deeper with convolutions［C］∥Proceedings of the IEEE Conference on computer vision and pattern recognition．Boston，MA：2015：1-9.
23	HUANG G， LIU Z， VAN DER MAATEN L，et al ．Densely connected convolutional networks［C］∥Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition （CVPR 2017）.Hawaii：IEEE，2017：2261-2269.

类型	图片数量	类型	图片数量
直行	2 934	直行或左转	2 466
左转	1 650	直行或右转	2 526
右转	1 956	直行或掉头	2 880

方法	平均 PSNR/db	每张平均耗时/s
人工修复	10.46	180
DeblurGAN模型	10.70	0.43

研究对象	识别准确率						平均准确率	平均 PSNR/db
研究对象	直行	左转	右转	直行或左转	直行或右转	直行或掉头	平均准确率	平均 PSNR/db
对照组（标签图像）	0.80	0.97	0.81	0.69	0.97	0.90	0.857	—
实验组1 （原始图像）	0.33	0.77	0.57	0.32	0.6	0.71	0.550	10.09
实验组2 （修复后图像）	0.81	0.96	0.82	0.71	0.97	0.91	0.863	17.91

参数	取值
平均PSNR值/db	10.164
标准差/db	3.542
最小PSNR值/db	3.022
下四分位PSNR值/db	7.773
中位PSNR值/db	10.039
上四分位PSNR值/db	12.644
最大PSNR值/dB	19.776

PSNR/db	等级	措施
0~5	差	及时维护
5~10	中	列入维护计划
10~15	良	增加巡查频率
>15	优	保持现状