Algorithm for Multiscale Residual Deformable Lung CT Image Registration

doi:10.12141/j.issn.1000-565X.230726

Abstract

Abstract:

The 4-dimensional CT (4D-CT) images of the lungs undergo large deformations due to respiration and heartbeat, and the scale of motion within the lungs may be larger than the structures of interest (blood vessels, airways, etc.) that the algorithm uses for the optimization process, which may result in the registration algorithms only aligning the obvious features such as blood vessels and airways. To address the problem of high variability of the aligned intensities for structures with large deformations such as the lung parenchyma contour, this paper proposed a multi-scale residual deformable lung CT image alignment algorithm framework based on unsupervised end-to-end deep learning. A multi-scale deep residual network in the form of an encoder-decoder structure was used as a generative model for the deformation field in the proposed registration framework, so as to enhance the feature representation, to increase the effective parameter utilization efficiency parameters and effectively improve the convergence ability of the network. A multi-resolution self-attentive fusion module was used to improve the network’s ability to perceive multi-scale information. And a hopping connection containing a feature correction extraction module was designed to selectively extract the feature maps output by the encoder and recalibrate them for the decoder to learn the alignment offsets. Finally, this paper compared the proposed alignment algorithm with traditional algorithms and the current state-of-the-art unsupervised alignment algorithms on the Dir-lab public dataset. The results show that, the target alignment error of the proposed registration algorithm framework on the Dir-lab public dataset can reach 1.44 mm ± 1.24 mm, which is better than traditional algorithms and the mainstream unsupervised alignment algorithm. In addition, the estimation of the dense deformation vector field takes less than 2.00 s with the control folding voxel less than 0.1%, indicating the great potential of the algorithm in studying time-sensitive lungs.

Key words: deep learning, lung CT image, image registration, unsupervised learning

CLC Number:

TP391

LIU Weipeng, LI Xu, REN Ziwen, QI Yedong. Algorithm for Multiscale Residual Deformable Lung CT Image Registration[J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 135-145.

Figures/Tables 7

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Table 2

Fig.5

References 25

1	AVANTS B B， TUSTISON N J， SONG G，et al ．A reproducible evaluation of ANTs similarity metric performance in brain image registration［J］．Neuroimage，2011，54（3）：2033-2044.
2	AVANTS B B， EPSTEIN C L， GROSSMAN M，et al ．Symmetric diffeomorphic image registration with cross-correlation：evaluating automated labeling of elderly and neurodegenerative brain［J］．Medical Image Analysis，2008，12（1）：26-41.
3	BEG M F， MILLER M I， TROUVÉ A，et al ．Computing large deformation metric mappings via geodesic flows of diffeomorphisms［J］．International Journal of Computer Vision，2005，61：139-157.
4	FU Y， LEI Y， WANG T，et al ．Deep learning in medical image registration：a review［J］．Physics in Medicine & Biology，2020，65（20）：20TR01/1-32.
5	CHEE E， WU Z ．AIRNet：self-supervised affine registration for 3D medical images using neural networks［EB/OL］．（2018-10-15）［2023-08-27］．.
6	CAO X， YANG J， ZHANG J，et al ．Deformable image registration based on similarity-steered CNN regression［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：300-308.
7	BIGALKE A， HANSEN L， MOK T C W，et al ．Unsupervised 3D registration through optimization-guided cyclical self-training［C］∥ GREENSPAN H，MADABHUSHI A，MOUSAVI P，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 26th International Conference．Vancouver：Springer，2023：677-687.
8	ZHANG J ．Inverse-consistent deep networks for unsupervised deformable image registration［EB/OL］．（2018-09-10）［2023-08-27］．.
9	BALAKRISHNAN G， ZHAO A， SABUNCU M R，et al ．Voxelmorph：a learning framework for deformable medical image registration［J］．IEEE Transactions on Medical Imaging，2019，38（8）：1788-1800.
10	彭昆，张桂梅，王杰，等．基于可变形卷积和多尺度特征聚焦的X线图像非刚性配准［J］．生物医学工程学杂志，2023，40（3）：492-498，507.
	PENG Kun， ZHANG Guimei， WANG Jie，et al ．Non-rigid registration for medical images based on deformable convolution and multi-scale feature focusing modules［J］．Journal of Biomedical Engineering，2023，40（3）：492-498，507.
11	HERING A， VAN GINNEKEN B， HELDMANN S ．mlVIRNet：multilevel variational image registration network［C］∥ SHEN D，LIU T，Peters T M，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 22nd International Conference．Shenzhen：Springer，2019：257-265.
12	ROHÉ M M， DATAR M， HEIMANN T，et al ．SVF-Net：learning deformable image registration using shape matching［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：266-274.
13	CHEN J， LI Y， DU Y，et al ．Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks［J］．Medical Physics，2020，47（12）：6366-6380.
14	HEINRICH M P， JENKINSON M， BRADY M，et al ．MRF-based deformable registration and ventilation estimation of lung CT［J］．IEEE Transactions on Medical Imaging，2013，32（7）：1239-1248.
15	JIANG Z， YIN F F， GE Y，et al ．A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration［J］．Physics in Medicine & Biology，2020，65（1）：015011/1-13.
16	DE VOS B D， BERENDSEN F F， VIERGEVER M A，et al ．A deep learning framework for unsupervised affine and deformable image registration［J］．Medical Image Analysis，2019，52：128-143.
17	YANG J， YANG J H， ZHAO F，et al ．An unsupervised multi-scale framework with attention-based network （MANet） for lung 4D-CT registration［J］．Physics in Medicine & Biology，2021，66（13）：135008/1-13.
18	HO T T， KIM W J， LEE C H，et al ．An unsupervised image registration method employing chest computed tomography images and deep neural networks［J］．Computers in Biology and Medicine，2023，154：106612/1-11.
19	JADERBERG M， SIMONYAN K， ZISSERMAN A.Spatial transformer networks［C］∥ Proceedings of the 28th International Conference on Neural Information Processing Systems．Cambridge：MIT Press，2015：2017-2025.
20	JIA X， THORLEY A， CHEN W，et al ．Learning a model-driven variational network for deformable image registration［J］．IEEE Transactions on Medical Imaging，2021，41（1）：199-212.
21	MOK T C W， CHUNG A C S C ．Fast symmetric diffeomorphic image registration with convolutional neural networks［C］∥ Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition．Seattle：IEEE，2020：4643-4652.
22	石磊，籍庆余，陈清威，等．视觉Transformer在医学图像分析中的应用研究综述［J］．计算机工程与应用，2023，59（8）：41-55.
	SHI Lei， JI Qingyu， CHEN Qingwei，et al ．Review of research on application of vision transformer in medical image analysis［J］．Computer Engineering and Applications，2023，59（8）：41-55.
23	DEVLIN J， CHANG M W， LEE K，et al ．BERT：pre-training of deep bidirectional transformers for language understanding［C］∥ Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies．Minneapolis：Association for Computational Linguistics，2019：4171-4186.
24	CASTILLO E， CASTILLO R， MARTINEZ J，et al ．Four-dimensional deformable image registration using trajectory modeling［J］．Physics in Medicine & Biology，2009，55（1）：305-327.
25	CASTILLO R， CASTILLO E， GUERRA R，et al ．A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets［J］．Physics in Medicine & Biology，2009，54（7）：1849-1870.

病例	TRE平均值±标准差
病例	初始值	ANT	DLIR	VM	LRN	MJ-CNN	文中算法
平均值	8.46 ± 6.58	1.49 ± 1.16	2.64 ± 4.32	2.43 ± 2.43	1.78 ± 1.56	1.66 ± 1.44	1.44 ± 1.24
病例1	3.89 ± 2.78	1.09 ± 0.75	1.27 ± 1.16	1.16 ± 0.80	1.14 ± 0.71	1.20 ± 0.63	1.21 ± 0.92
病例2	4.43 ± 3.90	1.04 ± 0.67	1.20 ± 1.12	1.09 ± 0.77	1.04 ± 1.13	1.13 ± 0.56	1.19 ± 0.83
病例3	6.94 ± 4.05	1.18 ± 0.78	1.48 ± 1.26	1.60 ± 1.02	1.44 ± 0.84	1.30 ± 0.70	1.31 ± 0.72
病例4	9.83 ± 4.86	1.50 ± 1.04	2.09 ± 1.93	2.46 ± 1.85	1.54 ± 1.20	1.55 ± 0.96	1.50 ± 0.95
病例5	7.48 ± 5.51	1.43 ± 1.19	1.95 ± 2.10	1.96 ± 1.66	1.60 ± 1.63	1.72 ± 1.28	1.51 ± 1.25
病例6	10.89 ± 6.97	1.50 ± 1.06	5.16 ± 7.09	2.85 ± 2.13	2.34 ± 1.29	2.02 ± 1.70	1.47 ± 1.08
病例7	11.03 ± 7.43	2.35 ± 1.93	3.05 ± 3.01	3.48 ± 2.75	1.80 ± 0.90	1.70 ± 1.03	1.45 ± 1.08
病例8	14.99 ± 9.01	1.51 ± 1.37	6.48 ± 5.37	5.17 ± 4.20	3.76 ± 2.52	2.64 ± 2.78	2.04 ± 3.46
病例9	7.92 ± 3.98	1.83 ± 1.27	2.10 ± 1.66	2.26 ± 1.56	1.62 ± 1.19	1.51 ± 0.94	1.29 ± 0.76
病例10	7.30 ± 6.35	1.42 ± 1.00	2.09 ± 2.24	2.29 ± 2.36	1.57 ± 1.54	1.79 ± 1.61	1.42 ± 1.39

算法	平均DSC	体素折叠率/%	GPU运行时间/s	CPU运行时间/s
ANTS	0.978	0.15		160.00
DLIR	0.971	0.09	3.13	42.00
VM	0.961	0.17	1.72	21.50
LRN	0.975	0.03	2.88	25.00
MJ-CNN	0.985	0.11	3.54	45.00
文中算法	0.988	0.07	1.94	24.00

[1]	HU Guanghua, DAI Zhigang, WANG Qinghui. Machining Feature Recognition Method of B-Rep Model Based on Graph Neural Network [J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(5): 20-31.
[2]	HU Xizhi, CUI Bofei, WANG Qin, et al. Visual SLAM Algorithm Based on Memory Parking Scene [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(6): 1-11.
[3]	LIU Hao, YUAN Hui, CHEN Chen, et al. Point Cloud Geometry Coding Framework Based on Sampling [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(6): 148-156.
[4]	YANG Chunling, LIANG Ziwen. Feature-Domain Proximal High-Dimensional Gradient Descent Network for Image Compressed Sensing [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(3): 119-130.
[5]	ZHENG Juanyi, DONG Jiahao, ZHANG Qingjue, et al. Reconfigurable Intelligence Surface Channel Estimation Algorithm Based on RDN [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(3): 102-111.
[6]	ZHOU Lang, FAN Kun, QU Hua, et al. Forest Fire Recognition by Improved EfficientNet-E Model Based on ECA Attention Mechanism [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(2): 42-49.
[7]	CHEN Qiong, FENG Yuan, LI Zhiqun, YANG Yong. Semantic-Visual Consistency Constraint Network for Zero-Shot Image Semantic Segmentation [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 41-50.
[8]	HU Guanghua, TU Qianxi. Surface Defect Detection Method for Industrial Products Based on Photometric Stereo and Dual Stream Feature Fusion Network [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 112-123.
[9]	LI Fang, GUO Weisen, ZHANG Ping, et al.. Prediction Technique for Remaining Useful Life of Bearing Based on Spatial-Temporal Dual Cell State [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 69-81.
[10]	SU Jindian, YU Shanshan, HONG Xiaobin. A Self-Supervised Pre-Training Method for Chinese Spelling Correction [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 90-98.
[11]	LI Jiachun, LI Bowen, LIN Weiwei. AdfNet: An Adaptive Deep Forgery Detection Network Based on Diverse Features [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 82-89.
[12]	GUO Enqiang, FU Xinsha. Dropped Object Detection Method Based on Feature Similarity Learning [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(6): 30-41.
[13]	ZHAO Jiandong, JIAO Lanxin, ZHAO Zhimin, et al. A Car-Following Model Driven by Combination of Theory and Data Considering Effects of Lane Change of Side Cars [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(6): 10-19.
[14]	YE Feng, CHEN Biao, LAI Yizong. Contrastive Knowledge Distillation Method Based on Feature Space Embedding [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 13-23.
[15]	ZHAO Rongchao, WU Baili, CHEN Zhuyun, et al. Graph Neural Network for Fault Diagnosis with Multi-Scale Time-Spatial Information Fusion Mechanism [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(12): 42-52.