Augmentation Method of Transportation Infrastructure Crack Images

doi:10.12141/j.issn.1000-565X.230350

Abstract

Abstract:

The crack detection model for transportation infrastructure based on deep learning relies on large-scale data for training. To address the problem of limited availability of diverse crack samples in specific transportation facility scenarios, this paper proposed a transportation infrastructure crack image augmentation method based on Pix2PixHD model. Initially, the Pix2PixHD model was used to establish a spatial mapping relationship between real crack images and annotated labels based on a small amount of collected crack image data. Subsequently, the objects in the label domain were edited to generate crack contours representing various forms, using methods such as label transfer from other datasets, manual editing, morphological dilation operations, and random superimposition operations. Finally, the edited label domain was transformed back to the image domain using the Pix2PixHD model, so as to achieve an adaptive augmentation of the transportation infrastructure crack dataset. This paper considered major materials and structures in transportation infrastructure (asphalt pavement, tunnel linings, and concrete structures) and conducted experiments using the GAPS384, Tunnel200, and DeepCrack datasets. Results demonstrate that the U-Net model trained on the augmented dataset achieves higher detection accuracy and is more likely to avoid local optima. Compared to the DCGAN model, the proposed method effectively controls crack morphology while maintaining the continuity of the crack skeleton, thereby enhancing the morphological diversity of the original crack dataset and improving the generalization capability of the detection model in specific transportation infrastructure scenarios.

Key words: transportation infrastructure, image generation, crack detection, generative adversarial network

CLC Number:

U418

JIANG Shengchuan, ZHONG Shan, WU Difei, LIU Chenglong. Augmentation Method of Transportation Infrastructure Crack Images[J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 146-158.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Table 3

Fig.6

Table 4

Fig.7

Fig.8

Table 5

Fig.9

References 26

1	李亦舜，刘成龙，曹静，等．基于深度学习的路面病害检测与时空追溯方法［J］．长安大学学报（自然科学版），2022，42（6）：53-65.
	LI Yi-shun， LIU Cheng-long， CAO Jing，et al ．Automatic tracking method of pavement performance decay based on deep learning［J］．Journal of Chang’an University （Natural Science Edition），2022，42（6）：53-65.
2	刘宇飞，樊健生，聂建国，等．结构表面裂缝数字图像法识别研究综述与前景展望［J］．土木工程学报，2021，54（6）：79-98.
	LIU Yufei， FAN Jiansheng， NIE Jianguo，et al ．Review and prospect of digital-image-based crack detection of structure surface［J］．China Civil Engineering Journal，2021，54（6）：79-98.
3	HU W B， WANG W D， AI C B，et al ．Machine vision-based surface crack analysis for transportation infrastructure［J］．Automation in Construction，2021，132：103973/1-22.
4	孙晓贺，施成华，刘凌晖，等．基于改进的种子填充算法的混凝土裂缝图像识别系统［J］．华南理工大学学报（自然科学版），2022，50（5）：127-136，146.
	SUN Xiaohe， SHI Chenghua， LIU Linghui，et al ．Concrete crack image recognition system based on improved seed filling algorithm［J］．Journal of South China University of Technology （Natural Science Edition），2022，50（5）：127-136，146.
5	刘晟，王卫星，曹霆，等．基于差分计盒维数及最大熵阈值的裂缝提取［J］．长安大学学报（自然科学版），2015，35（5）：13-21.
	LIU Sheng， WANG Wei-xing， CAO Ting，et al ．Road crack extraction based on differential box dimension and maximum entropy threshold［J］．Journal of Chang’an University （Natural Science Edition），2015，35（5）：13-21.
6	琚晓辉，徐凌．基于SVM-Adaboost裂缝图像分类方法研究［J］．公路交通科技，2017，34（11）：38-42.
	JU Xiao-hui， XU Ling ．Study on classification method of crack images based on SVM-Adaboost［J］．Journal of Highway and Transportation Research and Development，2017，34（11）：38-42.
7	邓露，褚鸿鹄，龙砺芝，等．基于深度学习的土木基础设施裂缝检测综述［J］．中国公路学报，2023，36（2）：1-21.
	DENG Lu， CHU Hong-hu， LONG Li-zhi，et al ．Review of deep learning-based crack detection for civil infrastructures［J］．China Journal of Highway and Transport，2023，36（2）：1-21.
8	张定军，廖明潮，高拉劳．基于GhostNet的轻量级桥梁裂缝图像语义分割算法［J］．公路，2023，68（4）：246-255.
	ZHANG Ding-jun， LIAO Ming-chao， GAO La-lao ．Light-weight bridge crack images segmentation algorithm based on GhostNet［J］．Highway，2023，68（4）：246-255.
9	邓治林，罗仁泽，费越，等．基于FE-Unet的机场道面裂缝检测［J］．光电子·激光，2023，34（1）：34-42.
	DENG Zhilin， LUO Renze， FEI Yue，et al ．Crack detection of airport pavement based on FE-Unet［J］．Journal of Optoelectronics·Laser，2023，34（1）：34-42.
10	LI Y S， CHE P Y， LIU C L，et al ．Cross-scene pavement distress detection by a novel transfer learning framework［J］．Computer-Aided Civil and Infrastructure Engineering，2021，36（11）：1398-1415.
11	LI Y S， LIU C L， SHEN Y，et al ．RoadID：a dedicated deep convolutional neural network for multipavement distress detection［J］．Journal of Transportation Engineering，Part B：Pavements，2021，147（4）：04021057/1-12.
12	李彦葓，李鹏飞，吕淼．基于深度学习的混凝土结构表面裂缝检测［J］．混凝土，2022（8）：187-192.
	LI Yanhong， LI Pengfei， Miao LÜ ．Surface crack detection of concrete structures based on deep learning［J］．Concrete，2022（8）：187-192.
13	许正森，雷相达，管海燕．多尺度局部特征增强Transformer道路裂缝检测模型［J］．中国图象图形学报，2023，28（4）：1019-1028.
	XU Zhengsen， LEI Xiangda， GUAN Haiyan ．Multi-scale local feature enhanced Transformer network for pavement crack detection［J］．Journal of Image and Graphics，2023，28（4）：1019-1028.
14	GOODFELLOW I J， POUGET-ABADIE J， MIRZA M，et al ．Generative adversarial nets［C］∥ GHAHRAMANI Z，WELLING M，CORTES C，et al．Advances in neural information processing systems 27．La Jolla：Neural Information Processing Systems，2014：2672-2680.
15	裴莉莉，孙朝云，孙静，等．基于DCGAN的路面裂缝图像生成方法［J］．中南大学学报（自然科学版），2021，52（11）：3899-3906.
	PEI Lili， SUN Chaoyun， SUN Jing，et al ．Generation method of pavement crack images based on deepconvolutional generative adversarial networks［J］．Journal of Central South University （Science and Technology），2021，52（11）：3899-3906.
16	ZHONG J T， HUYAN J， ZHANG W G，et al ．A deeper generative adversarial network for grooved cement concrete pavement crack detection［J］．Engineering Applications of Artificial Intelligence，2023，119：105808/1-13.
17	DONG J X， WANG N N， FANG H Y，et al ．Innovative method for pavement multiple damages segmentation and measurement by the Road-Seg-CapsNet of feature fusion［J］．Construction and Building Materials，2022，324：126719/1-20.
18	WANG T C， LIU M Y， ZHU J Y，et al ．High-resolution image synthesis and semantic manipulation with conditional GANs［C］∥ Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition．Salt Lake City：IEEE，2018：8798-8807.
19	WU D， GAN J H， ZHOU J X，et al ．Fine-grained semantic ethnic costume high-resolution image colorization with conditional GAN［J］．International Journal of Intelligent Systems，2022，37（5）：2952-2968.
20	LI J， CHEN Z， ZHAO X，et al ．MapGAN：an intelligent generation model for network tile maps［J］．Sensors，2020，20（11）：3119/1-20.
21	STRICKER R， AGANIAN D， SESSELMANN M，et al ．Road surface segmentation：pixel-perfect distress and object detection for road assessment［C］∥ Proceedings of 2021 IEEE the 17th International Conference on Automation Science and Engineering．Lyon：IEEE，2021：1789-1796.
22	ZHOU Q， QU Z， LI Y X，et al ．Tunnel crack detection with linear seam based on mixed attention and multiscale feature fusion［J］．IEEE Transactions on Instrumentation and Measurement，2022，71：5014711/1-11.
23	LIU Y H， YAO J， LU X H，et al ．DeepCrack：a deep hierarchical feature learning architecture for crack segmentation［J］．Neurocomputing，2019，338：139-153.
24	YANG F， ZHANG L， YU S J，et al ．Feature pyramid and hierarchical boosting network for pavement crack detection［J］．IEEE Transactions on Intelligent Transportation Systems，2020，21（4）：1525-1535.
25	RONNEBERGER O， FISCHER P， BROX T ．U-Net：convolutional networks for biomedical image segmentation［C］∥ Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention．Munich：Springer，2015：234-241.
26	KIM J， LEE J， PARK J，et al ．Pin the memory：learning to generalize semantic segmentation［C］∥ Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition．New Orleans：IEEE，2022：4340-4350.

训练集规模/幅	FID
训练集规模/幅	GAPS384	Tunnel200	DeepCrack
20	141.873	205.050	102.504
50	108.701	198.561	96.025
200	108.539	155.705	96.065
500	79.967		79.159

数据集	实验编号	训练集组成	测试集组成
GAPS384	1	50幅GAPS384图像	459幅GAPS384图像
	2	50幅GAPS384图像+1 000幅增广图像	459幅GAPS384图像
	3	50幅GAPS384图像+2 000幅增广图像	459幅GAPS384图像
	4	50幅GAPS384图像+3 000幅增广图像	459幅GAPS384图像
Tunnel200	5	20幅Tunnel200图像	180幅Tunnel200图像
	6	20幅Tunnel200图像+1 000幅增广图像	180幅Tunnel200图像
	7	20幅Tunnel200图像+2 000幅增广图像	180幅Tunnel200图像
	8	20幅Tunnel200图像+3 000幅增广图像	180幅Tunnel200图像
DeepCrack	9	50幅DeepCrack图像	487幅DeepCrack图像
	10	50幅DeepCrack图像+1 000幅增广图像	487幅DeepCrack图像
	11	50幅DeepCrack图像+2 000幅增广图像	487幅DeepCrack图像
	12	50幅DeepCrack图像+3 000幅增广图像	487幅DeepCrack图像

数据集	实验编号	MIoU/%	P/%	R/%
GAPS384	1	66.195	80.914	70.734
	2	66.765	81.715	71.251
	3	66.656	81.876	71.014
	4	66.857	81.822	71.342
Tunnel200	5	63.385	80.385	67.284
	6	68.123	81.766	73.791
	7	68.253	81.197	74.330
	8	67.955	81.513	73.670
DeepCrack	9	80.321	91.263	85.175
	10	82.755	93.062	86.908
	11	83.571	93.060	87.903
	12	83.666	92.898	88.145

[1]	ZHU Yu, XU Zhigang, ZHAO Xiangmo, et al. TsGAN-Based Automatic Generation Algorithm of Lane-Change Cut-in Test Scenarios on Expressways for Autonomous Vehicles [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(8): 76-88.
[2]	QIANG Ruiru, ZHAO Xiaoqiang. A Small Sample Rolling Bearing Fault Diagnosis Method Based on Gramian Angular Difference Field and Generative Adversarial Network [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 64-75.
[3]	ZHU Zhengyu, LUO Chao, HE Qianhua, et al. Multi-View Lip Motion and Voice Consistency Judgment Based on Lip Reconstruction and Three-Dimensional Coupled CNN [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 70-77.
[4]	Qi LIU Bin Yu. Pavement Crack Recognition Algorithm Based on Transposed CNN [J]. Journal of South China University of Technology(Natural Science Edition), 2021, 49(12): 124-132.
[5]	WEN Huiying ZHANG Weigang ZHAO Sheng. Vehicle Lane-Change Trajectory Prediction Model Based on Generative Adversarial Networks [J]. Journal of South China University of Technology (Natural Science Edition), 2020, 48(5): 32-40.
[6]	SUN Jifeng ZHU Yating WANG Kai. Motion Deblurring Based on DeblurGAN and Low Rank Decomposition [J]. Journal of South China University of Technology (Natural Science Edition), 2020, 48(1): 32-41,50.
[7]	Li Wei Huyan Ju Sha Ai-min Sun Zhao-yun Hao Xue-li . Pavement Crack Detection Based on Two - Scale Clustering Algorithm and 3D Data#br# [J]. Journal of South China University of Technology (Natural Science Edition), 2015, 43(8): 99-105.
[8]	Wang Wei- xing Wu Lin- chun. Extraction of Pavement Cracks Based on Valley Edge Detection of Fractional Integral [J]. Journal of South China University of Technology (Natural Science Edition), 2014, 42(1): 117-122.