基于偏移注意力机制和多特征融合的点云分类

doi:10.12141/j.issn.1000-565X.220545

摘要/Abstract

摘要：

三维点云由于受到雾、雨和雪等自然天气条件的影响较小而受到了广泛的关注，在交通、能源和医疗等多个领域得到了广泛的应用，其中点云分类旨在划分三维点云数据的类别，为不同领域决策者提供信息，实现解决方案的制订，对自动驾驶、故障诊断和医学影像分析等具有重要意义。点云分类的应用前景广阔，但目前仍面临着诸多挑战。由于点云的无序性、稀疏性和有限性等特点，传统的图像处理和计算机视觉方法难以直接应用于点云数据分析，直接利用卷积神经网络不能有效提取点云特征，部分模型的特征提取不够充分，局部和全局的信息未能有效的利用，可能丢失重要特征信息。针对上述问题，提出一种实现点云的局部和全局特征相结合的多特征融合模块，并结合偏移注意力机制嵌入多特征融合模块实现较深层次点云特征的提取，同时引入残差结构充分利用浅层提取的特征，防止网络过深导致浅层特征丢失。在ModelNet40和ScanObjectNN分类数据集上进行训练和测试，并对实验进行了消融研究和部分数据可视化。实验结果发现该模型在ModelNet40上的分类总体准确率为93.6%，与PointNet、LDGCNN和PCT等模型相比，分类总体准确率分别提高了4.4、0.7和0.4个百分点；在ScanObjectNN上的分类总体准确率为83.7%，与PointNet++和DGCNN相比，分类总体准确率分别提高了5.8和5.6个百分点，具有较高的准确率和鲁棒性。

关键词: 点云分类, 偏移注意力机制, 多特征融合, 残差网络

Abstract:

3D point cloud has received great attention due to the fact that they are less affected by natural weather conditions such as fog, rain and snow, and it is widely used in a variety of fields such as transportation, energy and healthcare. Point cloud classification aims to classify the categories of 3D point cloud data to provide information to decision makers in different fields and to enable the development of solutions, so it’s significant for automated driving, fault diagnosis and medical image analysis. The application of point cloud classification is promising, but it still faces many challenges. Due to the characteristics of point cloud such as disorder, sparseness and finiteness, traditional image processing and computer vision methods can not be directly applied to point cloud data analysis. The direct use of convolutional neural network can not effectively extract point cloud features; the feature extraction in some models is insufficient, and the local and global information is not effectively utilized, which may lead to the loss of important feature information. Aiming at these problems, a multi-feature fusion module combining local and global features of point cloud was proposed, and combined with the offset attention mechanism, the multi-feature fusion module was embedded to realize the extraction of deeper point cloud features. At the same time, the residual structure was introduced to make full use of the shallow extracted features to prevent the loss of shallow features caused by the overdepth of network. Training and testing were performed on ModelNet40 and ScanObjectNN classification datasets, and ablation studies and partial data visualization of the experiments were performed. The experimental results show that the overall classification accuracy of this model on ModelNet40 is 93.6%, which improves the overall classification accuracy by 4.4, 0.7 and 0.4 percentage points compared with PointNet, LDGCNN and PCT models, respectively. The overall accuracy of classification on ScanObjectNN is 83.7%, which is 5.8 and 5.6 percentage points higher than that of PointNet++ and DGCNN, respectively, with higher accuracy and robustness.

Key words: point cloud classification, offset attention mechanism, multi-feature fusion, residual network

中图分类号:

TP391

田晟, 宋霖, 赵凯龙. 基于偏移注意力机制和多特征融合的点云分类[J]. 华南理工大学学报(自然科学版), 2024, 52(1): 100-109.

TIAN Sheng, SONG Lin, ZHAO Kailong. Point Cloud Classification Based on Offset Attention Mechanism and Multi-Feature Fusion[J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(1): 100-109.

图/表 14

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

表3

图9

表4

表5

参考文献 29

1	SU H， MAJI S， KALOGERAKIS E，et al ．Multi-view convolutional neural networks for 3d shape recognition［C］∥ Proceedings of the IEEE International Conference on Computer Vision．Santiago：IEEE，2015：945-953.
2	MATURANA D， SCHERER S ．Voxnet：a 3d convolutional neural network for real-time object recognition［C］∥ Proceeding of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）．Hamburg：IEEE，2015：922-928.
3	ZHANG C， WAN H， SHEN X，et al ．PVT：point-voxel transformer for point cloud learning［J］．arXiv Preprint arXiv：，2021.
4	QI C R， SU H， MO K，et al ．Pointnet：deep learning on point sets for 3d classification and segmentation［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Honolulu：IEEE，2017：652-660.
5	QI C R， YI L， SU H，et al ．Pointnet++：deep hierarchical feature learning on point sets in a metric space［C］∥ Proceeding of Advances in Neural Information Processing Systems．Long Beach：Curran Associates Inc，2017：5105-5114.
6	LI Y， BU R， SUN M，et al ．Pointcnn：convolution on x-transformed points［C］．Proceeding of Advances in Neural Information Processing Systems．Montréal：Curran Associates Inc，2018：31.
7	WU W， QI Z， FUXIN L ．Pointconv：deep convolutional networks on 3d point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．Long Beach：IEEE，2019：9621-9630.
8	陈根，冯肖维．基于K最近邻局部点关系图卷积的点云分类［J］．应用激光，2022，42（2）：78-83.
	CHEN Gen， FENG Xiaowei ．Point cloud classification based on k-nearest neighbor local point relation graph convolution［J］．Applied Laser，2022，42（2）：78-83.
9	XU M， DING R， ZHAO H，et al ．Paconv：position adaptive convolution with dynamic kernel assembling on point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．Nashville：IEEE，2021：3173-3182.
10	WANG Y， SUN Y， LIU Z，et al ．Dynamic graph cnn for learning on point clouds［J］．Acm Transactions on Graphics （tog），2019，38（5）：1-12.
11	ZHANG K， HAO M， WANG J，et al ．Linked dynamic graph cnn：learning on point cloud via linking hierarchical features［J］．arXiv Preprint arXiv： 1904.10014，2019.
12	兰红，陈浩，张蒲芬．集图卷积和三维方向卷积的点云分类分割模型［J］．计算机工程与应用，2023，59（8）：182-191.
	LAN Hong， CHEN Hao， ZHANG Pufen ．Point cloud classification and segmentation model based on graph convolutionand 3D direction convolution［J］．Computer Engineering and Applications，2023，59（8）：182-191.
13	芦新宇，杨冰，叶海良，等．基于局部-非局部交互卷积的3D点云分类［J］．模式识别与人工智能，2022，35（2）：141-149.
	LU Xinyu， YANG Bing， YE Hailiang，et al ．3D point cloud classification based on local-nonlocal interactive convolution［J］．Pattern Recognition and Artificial Intelligence，2022，35（2）：141-149.
14	戴莫凡，邢帅，徐青，等．多特征融合与几何卷积的机载LiDAR点云地物分类［J］．中国图象图形学报，2022，27（2）：574-585.
	DAI Mofan， XING Shuai， XU Qing，et al ．Semantic segmentation of airborne LiDAR point cloud based on multi-feature fusion and geometric convolution［J］．Journal of Image and Graphics，2022，27（2）：574-585.
15	杜静，蔡国榕．多特征融合与残差优化的点云语义分割方法［J］．中国图象图形学报，2021，26（5）：1105-1116.
	DU Jing， CAI Guorong ．Point cloud semantic segmentation method based on multi-feature fusion and residual optimization［J］．Journal of Image and Graphics，2021，26（5）：1105-1116.
16	VASWANI A， SHAZEER N， PARMAR N，et al ．Attention is all you need［C］∥ Proceeding of Advances in Neural Information Processing Systems．Long Beach：NIPS，2017：5998-6008.
17	ZHAO H， JIANG L， JIA J，et al ．Point transformer［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Montreal：IEEE，2021：16259-16268.
18	CHEN C， FRAGONARA L Z， TSOURDOS A ．GAPointNet：graph attention based point neural network for exploiting local feature of point cloud［J］．Neurocomputing，2021，43：122-132.
19	GUO M H， CAI J X， LIU Z N，et al ．Pct：point cloud transformer［J］．Computational Visual Media，2021，7（2）：187-199.
20	陈涵娟，达飞鹏，盖绍彦．基于竞争注意力融合的深度三维点云分类网络［J］．浙江大学学报（工学版），2021，55（12）：2342-2351.
	CHEN Hanjuan， Feipeng DA， GAI Shaoyan ．Deep 3D point cloud classification network based on competitive attention fusion［J］．Journal of Zhejiang University（Engineering Science），2021，55（12）：2342-2351.
21	王利媛，付丽华．基于注意力机制点卷积网络的机载LiDAR点云分类［J］．激光与光电子学进展，2022，59（10）：456-465.
	WANG Liyuan， FU Lihua ．Airborne LiDAR point cloud classification based on attention mechanism point convolutional network［J］．Laser & Optoelectronics Progress，2022，59（10）：456-465.
22	XIANG T， ZHANG C， SONG Y，et al ．Walk in the cloud：learning curves for point clouds shape analysis［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Montreal：IEEE，2021：915-924.
23	RAN H， LIU J， WANG C ．Surface representation for point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．New Orleans：IEEE，2022：18942-18952.
24	CORDONNIER J B， LOUKAS A， JAGGI M ．On the relationship between self-attention and convolutional layers［J］．arXiv Preprint arXiv：，2019.
25	杜启亮，向照夷，田联房，等．用于动作识别的双流自适应注意力图卷积网络［J］．华南理工大学学报（自然科学版），2022，50（12）：20-29.
	DU Qiliang， XIANG Zhaoyi， TIAN Lianfang，et al ．Two-Stream adaptive attention graph convolutional networks for action recognition［J］．Journal of South China University of Technology（Natural Science Edition），2022，50（12）：20-29.
26	杨春玲，杨雅静．基于多尺度特征逐层融合深度神经网络的无参考图像质量评价方法［J］．华南理工大学学报（自然科学版），2022，50（4）：81-89.
	YANG Chunling， YANG Yajing ．A deep neural network based on layer-by-layer fusion of multi-scale features for no-reference image quality assessment［J］．Journal of South China University of Technology（Natural Science Edition），2022，50（4）：81-89.
27	WU Z， SONG S， KHOSLA A，et al ．3d shapenets：a deep representation for volumetric shapes［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Boston：IEEE，2015：1912-1920.
28	UY M A， PHAM Q H， HUA B S，et al ．Revisiting point cloud classification：a new benchmark dataset and classification model on real-world data［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Seoul：IEEE，2019：1588-1597.
29	QI C R， SU H， NIEßNER M，et al ．Volumetric and multi-view cnns for object classification on 3d data［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Las Vegas：IEEE，2016：5648-5656.

软件平台	配置情况
系统	Ubuntu 18.04
GPU	CUDA10.2+CuDNN
编程语言	Python3.8.3
深度学习库	Pytorch1.8.1
第三方库	h5py、mayavi、TensorboardX、OpenCV
可视化工具	Meshlab、CloudCompare

层级	模块	通道数变换
Make_layer1	Basic_bock×2	64，64
Make_layer2	Basic_bock×2	128，128
Make_layer3	Basic_bock×2	256，256
Make_layer4	Basic_bock×2	512，512
fc	—	1 024
pool	—	2 048
fc	—	512，256，40

模型	输入类型	点数	A_m/%	A_o/%
MVCNN^［1］	Views	1 k	—	90.1
VoxNet^［2］	Voxels	—	—	83.0
Subvolume^［29］	Voxels+Views	—	86.0	89.2
PVT^［3］	Voxels	—	—	94.0
PointNet^［4］	Points	1 k	86.0	89.2
PointNet++^［5］	Points+ normal	1 k	—	90.7
PointCNN^［6］	Points	1 k	88.1	92.2
Pointconv^［7］	Points+ normal	1 k	—	92.5
PAConv^［9］	Points	1 k	—	93.6
DGCNN^［10］	Points	1 k	90.2	92.2
LDGCNN^［11］	Points	1 k	90.3	92.9
CurveNet^［22］	Points	1 k	—	94.1
PCT^［19］	Points	1 k	—	93.2
本文	Points	1 k	90.7	93.6

模型	输入类型	点数	A_m/%	A_o/%
PointNet^［4］	Points	2k	63.4	68.2
PointNet++^［5］	Points+ normal	2k	75.4	77.9
PointCNN^［6］	Points	2k	75.1	78.5
Pointconv^［7］	Points+ normal	2k	75.3	77.4
DGCNN^［10］	Points	2k	73.6	78.1
本文	Points	1k	79.3	82.8
本文	Points	2k	80.9	83.7

模型	Multi-Feature Aggregation	Identity	Offset- Attention	A_m/%	A_o/%
A				86.0	89.2
B	√			88.7	92.3
C	√	√		90.2	93.1
D	√		√	89.3	92.1
E		√	√	86.9	91.6
F	√	√	√	90.7	93.6