Point Cloud Classification Based on Offset Attention Mechanism and Multi-Feature Fusion

doi:10.12141/j.issn.1000-565X.220545

Abstract

Abstract:

3D point cloud has received great attention due to the fact that they are less affected by natural weather conditions such as fog, rain and snow, and it is widely used in a variety of fields such as transportation, energy and healthcare. Point cloud classification aims to classify the categories of 3D point cloud data to provide information to decision makers in different fields and to enable the development of solutions, so it’s significant for automated driving, fault diagnosis and medical image analysis. The application of point cloud classification is promising, but it still faces many challenges. Due to the characteristics of point cloud such as disorder, sparseness and finiteness, traditional image processing and computer vision methods can not be directly applied to point cloud data analysis. The direct use of convolutional neural network can not effectively extract point cloud features; the feature extraction in some models is insufficient, and the local and global information is not effectively utilized, which may lead to the loss of important feature information. Aiming at these problems, a multi-feature fusion module combining local and global features of point cloud was proposed, and combined with the offset attention mechanism, the multi-feature fusion module was embedded to realize the extraction of deeper point cloud features. At the same time, the residual structure was introduced to make full use of the shallow extracted features to prevent the loss of shallow features caused by the overdepth of network. Training and testing were performed on ModelNet40 and ScanObjectNN classification datasets, and ablation studies and partial data visualization of the experiments were performed. The experimental results show that the overall classification accuracy of this model on ModelNet40 is 93.6%, which improves the overall classification accuracy by 4.4, 0.7 and 0.4 percentage points compared with PointNet, LDGCNN and PCT models, respectively. The overall accuracy of classification on ScanObjectNN is 83.7%, which is 5.8 and 5.6 percentage points higher than that of PointNet++ and DGCNN, respectively, with higher accuracy and robustness.

Key words: point cloud classification, offset attention mechanism, multi-feature fusion, residual network

CLC Number:

TP391

TIAN Sheng, SONG Lin, ZHAO Kailong. Point Cloud Classification Based on Offset Attention Mechanism and Multi-Feature Fusion[J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(1): 100-109.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 1

Table 2

Table 3

Fig.9

Table 4

Table 5

References 29

1	SU H， MAJI S， KALOGERAKIS E，et al ．Multi-view convolutional neural networks for 3d shape recognition［C］∥ Proceedings of the IEEE International Conference on Computer Vision．Santiago：IEEE，2015：945-953.
2	MATURANA D， SCHERER S ．Voxnet：a 3d convolutional neural network for real-time object recognition［C］∥ Proceeding of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems （IROS）．Hamburg：IEEE，2015：922-928.
3	ZHANG C， WAN H， SHEN X，et al ．PVT：point-voxel transformer for point cloud learning［J］．arXiv Preprint arXiv：，2021.
4	QI C R， SU H， MO K，et al ．Pointnet：deep learning on point sets for 3d classification and segmentation［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Honolulu：IEEE，2017：652-660.
5	QI C R， YI L， SU H，et al ．Pointnet++：deep hierarchical feature learning on point sets in a metric space［C］∥ Proceeding of Advances in Neural Information Processing Systems．Long Beach：Curran Associates Inc，2017：5105-5114.
6	LI Y， BU R， SUN M，et al ．Pointcnn：convolution on x-transformed points［C］．Proceeding of Advances in Neural Information Processing Systems．Montréal：Curran Associates Inc，2018：31.
7	WU W， QI Z， FUXIN L ．Pointconv：deep convolutional networks on 3d point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．Long Beach：IEEE，2019：9621-9630.
8	陈根，冯肖维．基于K最近邻局部点关系图卷积的点云分类［J］．应用激光，2022，42（2）：78-83.
	CHEN Gen， FENG Xiaowei ．Point cloud classification based on k-nearest neighbor local point relation graph convolution［J］．Applied Laser，2022，42（2）：78-83.
9	XU M， DING R， ZHAO H，et al ．Paconv：position adaptive convolution with dynamic kernel assembling on point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．Nashville：IEEE，2021：3173-3182.
10	WANG Y， SUN Y， LIU Z，et al ．Dynamic graph cnn for learning on point clouds［J］．Acm Transactions on Graphics （tog），2019，38（5）：1-12.
11	ZHANG K， HAO M， WANG J，et al ．Linked dynamic graph cnn：learning on point cloud via linking hierarchical features［J］．arXiv Preprint arXiv： 1904.10014，2019.
12	兰红，陈浩，张蒲芬．集图卷积和三维方向卷积的点云分类分割模型［J］．计算机工程与应用，2023，59（8）：182-191.
	LAN Hong， CHEN Hao， ZHANG Pufen ．Point cloud classification and segmentation model based on graph convolutionand 3D direction convolution［J］．Computer Engineering and Applications，2023，59（8）：182-191.
13	芦新宇，杨冰，叶海良，等．基于局部-非局部交互卷积的3D点云分类［J］．模式识别与人工智能，2022，35（2）：141-149.
	LU Xinyu， YANG Bing， YE Hailiang，et al ．3D point cloud classification based on local-nonlocal interactive convolution［J］．Pattern Recognition and Artificial Intelligence，2022，35（2）：141-149.
14	戴莫凡，邢帅，徐青，等．多特征融合与几何卷积的机载LiDAR点云地物分类［J］．中国图象图形学报，2022，27（2）：574-585.
	DAI Mofan， XING Shuai， XU Qing，et al ．Semantic segmentation of airborne LiDAR point cloud based on multi-feature fusion and geometric convolution［J］．Journal of Image and Graphics，2022，27（2）：574-585.
15	杜静，蔡国榕．多特征融合与残差优化的点云语义分割方法［J］．中国图象图形学报，2021，26（5）：1105-1116.
	DU Jing， CAI Guorong ．Point cloud semantic segmentation method based on multi-feature fusion and residual optimization［J］．Journal of Image and Graphics，2021，26（5）：1105-1116.
16	VASWANI A， SHAZEER N， PARMAR N，et al ．Attention is all you need［C］∥ Proceeding of Advances in Neural Information Processing Systems．Long Beach：NIPS，2017：5998-6008.
17	ZHAO H， JIANG L， JIA J，et al ．Point transformer［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Montreal：IEEE，2021：16259-16268.
18	CHEN C， FRAGONARA L Z， TSOURDOS A ．GAPointNet：graph attention based point neural network for exploiting local feature of point cloud［J］．Neurocomputing，2021，43：122-132.
19	GUO M H， CAI J X， LIU Z N，et al ．Pct：point cloud transformer［J］．Computational Visual Media，2021，7（2）：187-199.
20	陈涵娟，达飞鹏，盖绍彦．基于竞争注意力融合的深度三维点云分类网络［J］．浙江大学学报（工学版），2021，55（12）：2342-2351.
	CHEN Hanjuan， Feipeng DA， GAI Shaoyan ．Deep 3D point cloud classification network based on competitive attention fusion［J］．Journal of Zhejiang University（Engineering Science），2021，55（12）：2342-2351.
21	王利媛，付丽华．基于注意力机制点卷积网络的机载LiDAR点云分类［J］．激光与光电子学进展，2022，59（10）：456-465.
	WANG Liyuan， FU Lihua ．Airborne LiDAR point cloud classification based on attention mechanism point convolutional network［J］．Laser & Optoelectronics Progress，2022，59（10）：456-465.
22	XIANG T， ZHANG C， SONG Y，et al ．Walk in the cloud：learning curves for point clouds shape analysis［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Montreal：IEEE，2021：915-924.
23	RAN H， LIU J， WANG C ．Surface representation for point clouds［C］∥ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition．New Orleans：IEEE，2022：18942-18952.
24	CORDONNIER J B， LOUKAS A， JAGGI M ．On the relationship between self-attention and convolutional layers［J］．arXiv Preprint arXiv：，2019.
25	杜启亮，向照夷，田联房，等．用于动作识别的双流自适应注意力图卷积网络［J］．华南理工大学学报（自然科学版），2022，50（12）：20-29.
	DU Qiliang， XIANG Zhaoyi， TIAN Lianfang，et al ．Two-Stream adaptive attention graph convolutional networks for action recognition［J］．Journal of South China University of Technology（Natural Science Edition），2022，50（12）：20-29.
26	杨春玲，杨雅静．基于多尺度特征逐层融合深度神经网络的无参考图像质量评价方法［J］．华南理工大学学报（自然科学版），2022，50（4）：81-89.
	YANG Chunling， YANG Yajing ．A deep neural network based on layer-by-layer fusion of multi-scale features for no-reference image quality assessment［J］．Journal of South China University of Technology（Natural Science Edition），2022，50（4）：81-89.
27	WU Z， SONG S， KHOSLA A，et al ．3d shapenets：a deep representation for volumetric shapes［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Boston：IEEE，2015：1912-1920.
28	UY M A， PHAM Q H， HUA B S，et al ．Revisiting point cloud classification：a new benchmark dataset and classification model on real-world data［C］∥ Proceedings of the IEEE/CVF International Conference on Computer Vision．Seoul：IEEE，2019：1588-1597.
29	QI C R， SU H， NIEßNER M，et al ．Volumetric and multi-view cnns for object classification on 3d data［C］∥ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition．Las Vegas：IEEE，2016：5648-5656.

软件平台	配置情况
系统	Ubuntu 18.04
GPU	CUDA10.2+CuDNN
编程语言	Python3.8.3
深度学习库	Pytorch1.8.1
第三方库	h5py、mayavi、TensorboardX、OpenCV
可视化工具	Meshlab、CloudCompare

层级	模块	通道数变换
Make_layer1	Basic_bock×2	64，64
Make_layer2	Basic_bock×2	128，128
Make_layer3	Basic_bock×2	256，256
Make_layer4	Basic_bock×2	512，512
fc	—	1 024
pool	—	2 048
fc	—	512，256，40

模型	输入类型	点数	A_m/%	A_o/%
MVCNN^［1］	Views	1 k	—	90.1
VoxNet^［2］	Voxels	—	—	83.0
Subvolume^［29］	Voxels+Views	—	86.0	89.2
PVT^［3］	Voxels	—	—	94.0
PointNet^［4］	Points	1 k	86.0	89.2
PointNet++^［5］	Points+ normal	1 k	—	90.7
PointCNN^［6］	Points	1 k	88.1	92.2
Pointconv^［7］	Points+ normal	1 k	—	92.5
PAConv^［9］	Points	1 k	—	93.6
DGCNN^［10］	Points	1 k	90.2	92.2
LDGCNN^［11］	Points	1 k	90.3	92.9
CurveNet^［22］	Points	1 k	—	94.1
PCT^［19］	Points	1 k	—	93.2
本文	Points	1 k	90.7	93.6

模型	输入类型	点数	A_m/%	A_o/%
PointNet^［4］	Points	2k	63.4	68.2
PointNet++^［5］	Points+ normal	2k	75.4	77.9
PointCNN^［6］	Points	2k	75.1	78.5
Pointconv^［7］	Points+ normal	2k	75.3	77.4
DGCNN^［10］	Points	2k	73.6	78.1
本文	Points	1k	79.3	82.8
本文	Points	2k	80.9	83.7

模型	Multi-Feature Aggregation	Identity	Offset- Attention	A_m/%	A_o/%
A				86.0	89.2
B	√			88.7	92.3
C	√	√		90.2	93.1
D	√		√	89.3	92.1
E		√	√	86.9	91.6
F	√	√	√	90.7	93.6