多尺度残差可变形肺部CT图像配准算法

doi:10.12141/j.issn.1000-565X.230726

摘要/Abstract

摘要：

肺部4维CT（4D-CT）图像因受到呼吸、心跳的影响而发生较大的形变，肺内的运动尺度可能大于算法用于优化过程的感兴趣结构（血管、气道等），这可能导致配准算法仅对齐了血管、气道等明显特征。针对肺实质轮廓配准后强度差异性较大的问题，文中提出了以无监督端到端深度学习为基础的多尺度残差可变形肺部CT图像配准算法，使用编码器-解码器结构形式的多尺度深度残差网络作为形变向量场的生成模型，以增强特征表达能力，提高参数利用效率和网络收敛能力；通过多分辨率自注意力融合模块提高网络对多尺度信息的感知能力；设计包含特征校正提取模块的跳跃连接，以有选择地提取编码器输出的特征图，并在重新校准后供解码器学习对齐偏移。最后，在Dir-lab公共数据集上采用文中配准算法与传统算法、目前先进的无监督配准算法进行了比较实验。结果表明：所提出的配准算法在Dir-lab公共数据集上的目标配准误差可以达到1.44 mm ± 1.24 mm，优于传统算法和主流的无监督配准算法；在控制体素折叠率小于0.1%的情况下，估计密集变形向量场耗时小于2.00 s，表明文中算法在对时间敏感的肺部研究中有巨大潜力。

关键词: 深度学习, 肺部CT图像, 图像配准, 无监督学习

Abstract:

The 4-dimensional CT (4D-CT) images of the lungs undergo large deformations due to respiration and heartbeat, and the scale of motion within the lungs may be larger than the structures of interest (blood vessels, airways, etc.) that the algorithm uses for the optimization process, which may result in the registration algorithms only aligning the obvious features such as blood vessels and airways. To address the problem of high variability of the aligned intensities for structures with large deformations such as the lung parenchyma contour, this paper proposed a multi-scale residual deformable lung CT image alignment algorithm framework based on unsupervised end-to-end deep learning. A multi-scale deep residual network in the form of an encoder-decoder structure was used as a generative model for the deformation field in the proposed registration framework, so as to enhance the feature representation, to increase the effective parameter utilization efficiency parameters and effectively improve the convergence ability of the network. A multi-resolution self-attentive fusion module was used to improve the network’s ability to perceive multi-scale information. And a hopping connection containing a feature correction extraction module was designed to selectively extract the feature maps output by the encoder and recalibrate them for the decoder to learn the alignment offsets. Finally, this paper compared the proposed alignment algorithm with traditional algorithms and the current state-of-the-art unsupervised alignment algorithms on the Dir-lab public dataset. The results show that, the target alignment error of the proposed registration algorithm framework on the Dir-lab public dataset can reach 1.44 mm ± 1.24 mm, which is better than traditional algorithms and the mainstream unsupervised alignment algorithm. In addition, the estimation of the dense deformation vector field takes less than 2.00 s with the control folding voxel less than 0.1%, indicating the great potential of the algorithm in studying time-sensitive lungs.

Key words: deep learning, lung CT image, image registration, unsupervised learning

中图分类号:

TP391

刘卫朋, 李旭, 任子文, 祁业东. 多尺度残差可变形肺部CT图像配准算法[J]. 华南理工大学学报(自然科学版), 2024, 52(10): 135-145.

LIU Weipeng, LI Xu, REN Ziwen, QI Yedong. Algorithm for Multiscale Residual Deformable Lung CT Image Registration[J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 135-145.

图/表 7

图1

图2

图3

图4

表1

表2

图5

参考文献 25

1	AVANTS B B， TUSTISON N J， SONG G，et al ．A reproducible evaluation of ANTs similarity metric performance in brain image registration［J］．Neuroimage，2011，54（3）：2033-2044.
2	AVANTS B B， EPSTEIN C L， GROSSMAN M，et al ．Symmetric diffeomorphic image registration with cross-correlation：evaluating automated labeling of elderly and neurodegenerative brain［J］．Medical Image Analysis，2008，12（1）：26-41.
3	BEG M F， MILLER M I， TROUVÉ A，et al ．Computing large deformation metric mappings via geodesic flows of diffeomorphisms［J］．International Journal of Computer Vision，2005，61：139-157.
4	FU Y， LEI Y， WANG T，et al ．Deep learning in medical image registration：a review［J］．Physics in Medicine & Biology，2020，65（20）：20TR01/1-32.
5	CHEE E， WU Z ．AIRNet：self-supervised affine registration for 3D medical images using neural networks［EB/OL］．（2018-10-15）［2023-08-27］．.
6	CAO X， YANG J， ZHANG J，et al ．Deformable image registration based on similarity-steered CNN regression［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：300-308.
7	BIGALKE A， HANSEN L， MOK T C W，et al ．Unsupervised 3D registration through optimization-guided cyclical self-training［C］∥ GREENSPAN H，MADABHUSHI A，MOUSAVI P，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 26th International Conference．Vancouver：Springer，2023：677-687.
8	ZHANG J ．Inverse-consistent deep networks for unsupervised deformable image registration［EB/OL］．（2018-09-10）［2023-08-27］．.
9	BALAKRISHNAN G， ZHAO A， SABUNCU M R，et al ．Voxelmorph：a learning framework for deformable medical image registration［J］．IEEE Transactions on Medical Imaging，2019，38（8）：1788-1800.
10	彭昆，张桂梅，王杰，等．基于可变形卷积和多尺度特征聚焦的X线图像非刚性配准［J］．生物医学工程学杂志，2023，40（3）：492-498，507.
	PENG Kun， ZHANG Guimei， WANG Jie，et al ．Non-rigid registration for medical images based on deformable convolution and multi-scale feature focusing modules［J］．Journal of Biomedical Engineering，2023，40（3）：492-498，507.
11	HERING A， VAN GINNEKEN B， HELDMANN S ．mlVIRNet：multilevel variational image registration network［C］∥ SHEN D，LIU T，Peters T M，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 22nd International Conference．Shenzhen：Springer，2019：257-265.
12	ROHÉ M M， DATAR M， HEIMANN T，et al ．SVF-Net：learning deformable image registration using shape matching［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：266-274.
13	CHEN J， LI Y， DU Y，et al ．Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks［J］．Medical Physics，2020，47（12）：6366-6380.
14	HEINRICH M P， JENKINSON M， BRADY M，et al ．MRF-based deformable registration and ventilation estimation of lung CT［J］．IEEE Transactions on Medical Imaging，2013，32（7）：1239-1248.
15	JIANG Z， YIN F F， GE Y，et al ．A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration［J］．Physics in Medicine & Biology，2020，65（1）：015011/1-13.
16	DE VOS B D， BERENDSEN F F， VIERGEVER M A，et al ．A deep learning framework for unsupervised affine and deformable image registration［J］．Medical Image Analysis，2019，52：128-143.
17	YANG J， YANG J H， ZHAO F，et al ．An unsupervised multi-scale framework with attention-based network （MANet） for lung 4D-CT registration［J］．Physics in Medicine & Biology，2021，66（13）：135008/1-13.
18	HO T T， KIM W J， LEE C H，et al ．An unsupervised image registration method employing chest computed tomography images and deep neural networks［J］．Computers in Biology and Medicine，2023，154：106612/1-11.
19	JADERBERG M， SIMONYAN K， ZISSERMAN A.Spatial transformer networks［C］∥ Proceedings of the 28th International Conference on Neural Information Processing Systems．Cambridge：MIT Press，2015：2017-2025.
20	JIA X， THORLEY A， CHEN W，et al ．Learning a model-driven variational network for deformable image registration［J］．IEEE Transactions on Medical Imaging，2021，41（1）：199-212.
21	MOK T C W， CHUNG A C S C ．Fast symmetric diffeomorphic image registration with convolutional neural networks［C］∥ Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition．Seattle：IEEE，2020：4643-4652.
22	石磊，籍庆余，陈清威，等．视觉Transformer在医学图像分析中的应用研究综述［J］．计算机工程与应用，2023，59（8）：41-55.
	SHI Lei， JI Qingyu， CHEN Qingwei，et al ．Review of research on application of vision transformer in medical image analysis［J］．Computer Engineering and Applications，2023，59（8）：41-55.
23	DEVLIN J， CHANG M W， LEE K，et al ．BERT：pre-training of deep bidirectional transformers for language understanding［C］∥ Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies．Minneapolis：Association for Computational Linguistics，2019：4171-4186.
24	CASTILLO E， CASTILLO R， MARTINEZ J，et al ．Four-dimensional deformable image registration using trajectory modeling［J］．Physics in Medicine & Biology，2009，55（1）：305-327.
25	CASTILLO R， CASTILLO E， GUERRA R，et al ．A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets［J］．Physics in Medicine & Biology，2009，54（7）：1849-1870.

病例	TRE平均值±标准差
病例	初始值	ANT	DLIR	VM	LRN	MJ-CNN	文中算法
平均值	8.46 ± 6.58	1.49 ± 1.16	2.64 ± 4.32	2.43 ± 2.43	1.78 ± 1.56	1.66 ± 1.44	1.44 ± 1.24
病例1	3.89 ± 2.78	1.09 ± 0.75	1.27 ± 1.16	1.16 ± 0.80	1.14 ± 0.71	1.20 ± 0.63	1.21 ± 0.92
病例2	4.43 ± 3.90	1.04 ± 0.67	1.20 ± 1.12	1.09 ± 0.77	1.04 ± 1.13	1.13 ± 0.56	1.19 ± 0.83
病例3	6.94 ± 4.05	1.18 ± 0.78	1.48 ± 1.26	1.60 ± 1.02	1.44 ± 0.84	1.30 ± 0.70	1.31 ± 0.72
病例4	9.83 ± 4.86	1.50 ± 1.04	2.09 ± 1.93	2.46 ± 1.85	1.54 ± 1.20	1.55 ± 0.96	1.50 ± 0.95
病例5	7.48 ± 5.51	1.43 ± 1.19	1.95 ± 2.10	1.96 ± 1.66	1.60 ± 1.63	1.72 ± 1.28	1.51 ± 1.25
病例6	10.89 ± 6.97	1.50 ± 1.06	5.16 ± 7.09	2.85 ± 2.13	2.34 ± 1.29	2.02 ± 1.70	1.47 ± 1.08
病例7	11.03 ± 7.43	2.35 ± 1.93	3.05 ± 3.01	3.48 ± 2.75	1.80 ± 0.90	1.70 ± 1.03	1.45 ± 1.08
病例8	14.99 ± 9.01	1.51 ± 1.37	6.48 ± 5.37	5.17 ± 4.20	3.76 ± 2.52	2.64 ± 2.78	2.04 ± 3.46
病例9	7.92 ± 3.98	1.83 ± 1.27	2.10 ± 1.66	2.26 ± 1.56	1.62 ± 1.19	1.51 ± 0.94	1.29 ± 0.76
病例10	7.30 ± 6.35	1.42 ± 1.00	2.09 ± 2.24	2.29 ± 2.36	1.57 ± 1.54	1.79 ± 1.61	1.42 ± 1.39

算法	平均DSC	体素折叠率/%	GPU运行时间/s	CPU运行时间/s
ANTS	0.978	0.15		160.00
DLIR	0.971	0.09	3.13	42.00
VM	0.961	0.17	1.72	21.50
LRN	0.975	0.03	2.88	25.00
MJ-CNN	0.985	0.11	3.54	45.00
文中算法	0.988	0.07	1.94	24.00

[1]	胡广华, 代志刚, 王清辉. 基于图神经网络的B-Rep模型加工特征识别方法[J]. 华南理工大学学报(自然科学版), 2025, 53(5): 20-31.
[2]	胡习之, 崔博非, 王琴, 等. 基于记忆泊车场景的视觉SLAM算法[J]. 华南理工大学学报(自然科学版), 2024, 52(6): 1-11.
[3]	刘昊, 元辉, 陈晨, 等. 基于采样的点云几何编码框架[J]. 华南理工大学学报(自然科学版), 2024, 52(6): 148-156.
[4]	杨春玲, 梁梓文. 特征域近端高维梯度下降图像压缩感知重构网络[J]. 华南理工大学学报(自然科学版), 2024, 52(3): 119-130.
[5]	郑娟毅, 董嘉豪, 张庆珏, 等. 基于残差密集网络的智能超表面信道估计算法[J]. 华南理工大学学报(自然科学版), 2024, 52(3): 102-111.
[6]	周浪, 樊坤, 瞿华, 等. 基于ECA注意力机制改进的EfficientNet-E模型的森林火灾识别[J]. 华南理工大学学报(自然科学版), 2024, 52(2): 42-49.
[7]	陈琼, 冯媛, 李志群, 杨咏. 基于语义-视觉一致性约束的零样本图像语义分割网络[J]. 华南理工大学学报(自然科学版), 2024, 52(10): 41-50.
[8]	胡广华, 涂千禧. 基于光度立体和双流特征融合网络的工业产品表面缺陷检测方法[J]. 华南理工大学学报(自然科学版), 2024, 52(10): 112-123.
[9]	李方, 郭炜森, 张平, 等. 基于时空双细胞状态的轴承剩余使用寿命预测方法[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 69-81.
[10]	苏锦钿, 余珊珊, 洪晓斌. 一种面向中文拼写纠错的自监督预训练方法[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 90-98.
[11]	李家春, 李博文, 林伟伟. AdfNet：一种基于多样化特征的自适应深度伪造检测网络[J]. 华南理工大学学报(自然科学版), 2023, 51(9): 82-89.
[12]	郭恩强, 符锌砂. 基于特征相似性学习的抛洒物检测方法[J]. 华南理工大学学报(自然科学版), 2023, 51(6): 30-41.
[13]	赵建东, 焦岚馨, 赵志敏, 等. 考虑侧向车换道影响的理论和数据组合驱动的车辆跟驰模型[J]. 华南理工大学学报(自然科学版), 2023, 51(6): 10-19.
[14]	叶峰, 陈彪, 赖乙宗. 基于特征空间嵌入的对比知识蒸馏算法[J]. 华南理工大学学报(自然科学版), 2023, 51(5): 13-23.
[15]	赵荣超, 吴百礼, 陈祝云, 等. 多尺度时空信息融合驱动的图神经网络故障诊断方法[J]. 华南理工大学学报(自然科学版), 2023, 51(12): 42-52.