基于特征相似性学习的抛洒物检测方法

doi:10.12141/j.issn.1000-565X.220604

摘要/Abstract

摘要：

针对当前以目标检测为核心的抛洒物检测算法无法识别“未知类别”的缺陷，以抛洒物引发外观特征变化的视角切入，提出基于特征相似性学习的抛洒物检测方法。首先，在抛洒物体过程中采集参考图像和待检图像，通过参数共享的孪生卷积神经网络得到两张图像的外观特征，然后利用欧式距离等特征相似性函数计算图像区域之间的特征变化并得到欧式距离热力图，最后经阈值筛选得到抛洒物检测结果。为了提升算法对光照等噪声的抗干扰能力，提出全新的注意力掩膜单元，并通过构建长跨度上下文信息和强监督学习的方式提升注意力掩膜的语义判别性能，引导特征响应聚焦于抛洒物引起的外观变化，同时忽略噪声产生的扰动，最终解决噪声干扰和抛洒物产生的特征缠绕问题。为了验证方法的有效性，本研究在真实高速公路场景下进行视频影像数据采集、标注、构建成标准数据集。结果表明：注意力掩膜单元有效提升了特征的语义判别性能，大幅度提高抛洒物检测精度，其中调和均值 $F 1$ 提高6.4个百分点，同时算法运行速度稳定在30帧/s，满足实时性需求；利用特征序列状态转移方式构建的长跨度上下文信息更有利于注意力掩膜聚焦抛洒物特征信息，抗噪声干扰能力更强；通过强监督学习得到的注意力掩膜轮廓更为准确，模型精度更高。

关键词: 抛洒物识别, 深度学习, 特征相似性学习, 注意力机制, 上下文信息

Abstract:

To overcome the limitation that the existing dropped object detection methods cannot identify the “unknown category”, this study proposed a dropped object detection architecture based on feature similarity learning. Firstly, the reference image and the query image to be detected were obtained during the dropping process. The appearance features were extracted through a weight-shared siamese convolutional network. Then, Euclidean distance was used to measure dissimilarities between features of reference image and query image. Finally, dropped objects were detected by selecting the pixel from the distance map whose distance value was larger than the fixed threshold. In order to improve its robustness to noise such as illumination change, this paper proposed a novel attention mask unit. And the semantic discriminativeness of the mask was improved through constructing the long-span contextual information and strong supervised learning method. This finally guides the feature response to focus on the appearance changes caused by the dropped objects while ignore the disturbance caused by noise, and solves the problem of feature entanglement between noise and the dropped objects. In order to verify the effectiveness of the method, this study collected data in a real highway scene and built a standard dataset. The results show that the attention mask unit effectively improves the semantic discriminative of features and greatly improves the accuracy of dropped object detection, which achieves F₁ an improvement of 6.4 percentage points. Meanwhile, the algorithm reaches in 30 FPS, which can be performed in real-time. The long-span context information constructed by the feature sequence state transition method is more conducive to attention mask focusing on the projectile feature information, and has stronger anti-noise interference ability. The attention mask contour obtained by strongly supervised learning is more accurate and the model accuracy is higher.

Key words: dropped object detection, deep learning, feature similarity learning, attention mechanism, context information

中图分类号:

U495

郭恩强, 符锌砂. 基于特征相似性学习的抛洒物检测方法[J]. 华南理工大学学报(自然科学版), 2023, 51(6): 30-41.

GUO Enqiang, FU Xinsha. Dropped Object Detection Method Based on Feature Similarity Learning[J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(6): 30-41.

图/表 2

参考文献 23

1	蒋来．浅谈高速公路抛洒物危害与对策［J］．道路交通管理，2021（4）：36-37.
	JIANG Lai ．A brief discussion on the hazards and countermeasures of abandoned objects on highways ［J］．Road Traffic Management，2021（4）：36-37.
2	李清瑶，邹皓，赵群，等．基于帧间差分自适应法的车辆抛洒物检测［J］．长春理工大学学报（自然科学版），2018，41（4）：108-113.
	LI Qingyao， ZOU Hao， ZHAO Qun，et al ．Vehicle throwing detection based on inter-frame difference adaptive method ［J］．Journal of Changchun University of Science and Technology （Natural Science Edition），2018，41（4）：108-113.
3	DIN M， BASHIR A， BASIT A，et al ．Abandoned object detection using frame differencing and background subtraction ［J］．International Journal of Applied Mathematics and Computer Science and Applications，2020，11（7）：676-681.
4	ZENG Y， LAN J， RAN B，et al ．A novel abandoned object detection system based on three-dimensional image information ［J］．Sensors，2015，15（3）：6885-6904.
5	夏莹杰，欧阳聪宇．面向高速公路抛洒物检测的动态背景建模方法［J］．浙江大学学报（工学版），2020，54（7）：1249-1255.
	XIA Yingjie， OUYANG Congyu ．Dynamic image background modeling method for detecting abandoned objects in highway ［J］．Journal of Zhejiang University （Engineering Science），2020，54（7）：1249-1255.
6	FU H， XIANG M， MA H，et al ．Abandoned object detection in highway scene ［C］∥ Proceedings of the 2011 6th International Conference on Pervasive Computing and Applications．Port Elizabeth：IEEE，2011：117-121.
7	汪贵平，马力旺，郭璐，等．高速公路抛洒物事件图像检测算法［J］．长安大学学报（自然科学版），2017，37（5）：81-88.
	WANG Guiping， MA Liwang， GUO Lu，et al ．Image detection algorithm for incident of discarded things in highway［J］．Journal of Chang’an University （Natural Science Edition），2017，37（5）：81-88.
8	金瑶，张锐，尹东．城市道路视频中小像素目标检测［J］．光电工程，2019，46（9）：76-83.
	JIN Yao， ZHANG Rui， YIN Dong ．Object detection for small pixel in urban roads videos ［J］．Optoelectronic Engineering，2019，46（9）：76-83.
9	章悦，张亮，谢非，等．基于实例分割模型优化的道路抛洒物检测算法［J］．计算机应用，2021，41（11）：3228-3233.
	ZHANG Yue， ZHANG Liang， XIE Fei，et al ．Road abandoned object detection algorithm based on optimized instance segmentation model［J］．Journal of Computer Applications，2021，41（11）：3228-3233.
10	BELL S， ZITNICK C L， BALA K，et al ．Inside-outside net： detecting objects in context with skip pooling and recurrent neural networks ［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Las Vegas：IEEE，2016：2874-2883.
11	WOO S， PARK J， LEE J-Y，et al ．CBAM：convolutional block attention module ［C］∥ Proceedings of the European Conference on Computer Vision （ECCV）．Munich：Springer，2018：3-19.
12	PARK J，WOO S， LEE J-Y，et al ．A simple and light-weight attention module for convolutional neural networks［J］．International Journal of Computer Vision，2020，128（4）：783-798.
13	CHUNG J， GULCEHRE C， CHO K，et al ．Empirical evaluation of gated recurrent neural networks on sequence modeling ［J］．arXiv preprint arXiv：，2014.
14	BOUTROS F， DAMER N， KIRCHBUCHNER F，et al ．Elasticface：elastic margin loss for deep face recognition ［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）．New Orleans：IEEE，2022：1577-1586.
15	JIN X， HE T， ZHENG K，et al ．Cloth-changing person re-identification from a single image with gait prediction and regularization ［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．New Orleans：IEEE，2022：14278-14287.
16	SHEN Q， QIAO L， GUO J，et al ．Unsupervised learning of accurate Siamese tracking ［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．New Orleans：IEEE，2022：8101-8110.
17	PASZKE A， GROSS S， MASSA F，et al ．PyTorch：an imperative style，high-performance deep learning library［C］∥ Proceeding of NeurIPS 2019．Vancouver：［s.n.］ 2019.
18	HE K， ZHANG X， REN S，et al ．Deep residual learning for image recognition ［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Las Vegas：IEEE，2016：770-778.
19	LONG J， SHELHAMER E， DARRELL T ．Fully convolutional networks for semantic segmentation ［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Boston：IEEE，2015：3431-3440.
20	RONNEBERGER O， FISCHER P， BROX T ．U-Net：convolutional networks for biomedical image segmentation ［C］∥ Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention．Munich：Springer，2015：234-241.
21	CHEN L-C， ZHU Y， PAPANDREOU G，et al ．Encoder-decoder with atrous separable convolution for semantic image segmentation ［C］∥ Proceedings of the European Conference on Computer Vision （ECCV）．Munich：Springer，2018：801-818.
22	LIN G， LIU F， MILAN A，et al ．RefineNet：multi-path refinement networks for dense prediction ［J］．IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，42（5）：1228-1242.
23	YU F， KOLTUN V， FUNKHOUSER T ．Dilated residual networks ［C］∥ Proceedings of the IEEE conference on Computer Vision and Pattern Recognition．Honolulu：IEEE，2017：472-480.

[1]	杨春玲, 梁梓文. 特征域近端高维梯度下降图像压缩感知重构网络[J]. 华南理工大学学报(自然科学版), 2024, 52(3): 119-130.
[2]	郑娟毅, 董嘉豪, 张庆珏, 等. 基于残差密集网络的智能超表面信道估计算法[J]. 华南理工大学学报(自然科学版), 2024, 52(3): 102-111.
[3]	周浪, 樊坤, 瞿华, 等. 基于ECA注意力机制改进的EfficientNet-E模型森林火灾识别研究[J]. 华南理工大学学报(自然科学版), 2024, 52(2): 42-49.
[4]	田晟, 宋霖, 赵凯龙. 基于偏移注意力机制和多特征融合的点云分类[J]. 华南理工大学学报(自然科学版), 2024, 52(1): 100-109.
[5]	李海燕, 尹浩林, 李鹏, 等. 基于密集特征推理及混合损失函数的修复算法[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 99-109.
[6]	李方, 郭炜森, 张平, 等. 基于时空双细胞状态的轴承剩余使用寿命预测方法[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 69-81.
[7]	苏锦钿, 余珊珊, 洪晓斌. 一种面向中文拼写纠错的自监督预训练方法[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 90-98.
[8]	李家春, 李博文, 林伟伟. AdfNet：一种基于多样化特征的自适应深度伪造检测网络[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 82-89.
[9]	赵建东, 焦岚馨, 赵志敏, 等. 考虑侧向车换道影响的理论和数据组合驱动的车辆跟驰模型[J]. 华南理工大学学报(自然科学版), 2023, 51(6): 10-19.
[10]	陆璐, 赖锦雄. 基于胶囊网络和注意力机制的智能合约漏洞检测方法[J]. 华南理工大学学报(自然科学版), 2023, 51(5): 36-44.
[11]	叶峰, 陈彪, 赖乙宗. 基于特征空间嵌入的对比知识蒸馏算法[J]. 华南理工大学学报(自然科学版), 2023, 51(5): 13-23.
[12]	刘宇鹏, 张雷. 融合遗忘和知识点重要度的认知诊断模型[J]. 华南理工大学学报(自然科学版), 2023, 51(5): 54-62.
[13]	赵荣超, 吴百礼, 陈祝云, 温楷儒, 张绍辉, 李巍华. 多尺度时空信息融合驱动的图神经网络故障诊断方法[J]. 华南理工大学学报(自然科学版), 2023, 51(12): 42-52.
[14]	侯力玮, 王恒升, 邹浩然. 基于深度学习的玻璃基板铲起过程作用力预测[J]. 华南理工大学学报(自然科学版), 2022, 50(8): 71-81.
[15]	莫建文, 朱彦桥, 袁华, 等. 基于神经元正则和资源释放的增量学习[J]. 华南理工大学学报(自然科学版), 2022, 50(6): 71-79,90.