收稿日期: 2023-11-22
网络出版日期: 2024-01-31
基金资助
国家重点研发计划项目(2020YFB1313703);国家自然科学基金资助项目(62027813);河北省重点研发计划项目(21372003D);河北省自然科学基金资助项目(F2022202054)
Algorithm for Multiscale Residual Deformable Lung CT Image Registration
Received date: 2023-11-22
Online published: 2024-01-31
Supported by
the National Key R & D Program of China(2020YFB1313703);the National Natural Science Foundation of China(62027813);the Key R & D Program of Hebei Province(21372003D);the Natural Science Foundation of Hebei Province(F2022202054)
肺部4维CT(4D-CT)图像因受到呼吸、心跳的影响而发生较大的形变,肺内的运动尺度可能大于算法用于优化过程的感兴趣结构(血管、气道等),这可能导致配准算法仅对齐了血管、气道等明显特征。针对肺实质轮廓配准后强度差异性较大的问题,文中提出了以无监督端到端深度学习为基础的多尺度残差可变形肺部CT图像配准算法,使用编码器-解码器结构形式的多尺度深度残差网络作为形变向量场的生成模型,以增强特征表达能力,提高参数利用效率和网络收敛能力;通过多分辨率自注意力融合模块提高网络对多尺度信息的感知能力;设计包含特征校正提取模块的跳跃连接,以有选择地提取编码器输出的特征图,并在重新校准后供解码器学习对齐偏移。最后,在Dir-lab公共数据集上采用文中配准算法与传统算法、目前先进的无监督配准算法进行了比较实验。结果表明:所提出的配准算法在Dir-lab公共数据集上的目标配准误差可以达到1.44 mm ± 1.24 mm,优于传统算法和主流的无监督配准算法;在控制体素折叠率小于0.1%的情况下,估计密集变形向量场耗时小于2.00 s,表明文中算法在对时间敏感的肺部研究中有巨大潜力。
刘卫朋 , 李旭 , 任子文 , 祁业东 . 多尺度残差可变形肺部CT图像配准算法[J]. 华南理工大学学报(自然科学版), 2024 , 52(10) : 135 -145 . DOI: 10.12141/j.issn.1000-565X.230726
The 4-dimensional CT (4D-CT) images of the lungs undergo large deformations due to respiration and heartbeat, and the scale of motion within the lungs may be larger than the structures of interest (blood vessels, airways, etc.) that the algorithm uses for the optimization process, which may result in the registration algorithms only aligning the obvious features such as blood vessels and airways. To address the problem of high variability of the aligned intensities for structures with large deformations such as the lung parenchyma contour, this paper proposed a multi-scale residual deformable lung CT image alignment algorithm framework based on unsupervised end-to-end deep learning. A multi-scale deep residual network in the form of an encoder-decoder structure was used as a generative model for the deformation field in the proposed registration framework, so as to enhance the feature representation, to increase the effective parameter utilization efficiency parameters and effectively improve the convergence ability of the network. A multi-resolution self-attentive fusion module was used to improve the network’s ability to perceive multi-scale information. And a hopping connection containing a feature correction extraction module was designed to selectively extract the feature maps output by the encoder and recalibrate them for the decoder to learn the alignment offsets. Finally, this paper compared the proposed alignment algorithm with traditional algorithms and the current state-of-the-art unsupervised alignment algorithms on the Dir-lab public dataset. The results show that, the target alignment error of the proposed registration algorithm framework on the Dir-lab public dataset can reach 1.44 mm ± 1.24 mm, which is better than traditional algorithms and the mainstream unsupervised alignment algorithm. In addition, the estimation of the dense deformation vector field takes less than 2.00 s with the control folding voxel less than 0.1%, indicating the great potential of the algorithm in studying time-sensitive lungs.
Key words: deep learning; lung CT image; image registration; unsupervised learning
| 1 | AVANTS B B, TUSTISON N J, SONG G,et al .A reproducible evaluation of ANTs similarity metric performance in brain image registration[J].Neuroimage,2011,54(3):2033-2044. |
| 2 | AVANTS B B, EPSTEIN C L, GROSSMAN M,et al .Symmetric diffeomorphic image registration with cross-correlation:evaluating automated labeling of elderly and neurodegenerative brain[J].Medical Image Analysis,2008,12(1):26-41. |
| 3 | BEG M F, MILLER M I, TROUVé A,et al .Computing large deformation metric mappings via geodesic flows of diffeomorphisms[J].International Journal of Computer Vision,2005,61:139-157. |
| 4 | FU Y, LEI Y, WANG T,et al .Deep learning in medical image registration:a review[J].Physics in Medicine & Biology,2020,65(20):20TR01/1-32. |
| 5 | CHEE E, WU Z .AIRNet:self-supervised affine registration for 3D medical images using neural networks[EB/OL].(2018-10-15)[2023-08-27].. |
| 6 | CAO X, YANG J, ZHANG J,et al .Deformable image registration based on similarity-steered CNN regression[C]∥ DESCOTEAUX M,MAIER-HEIN L,FRANZ A,et al.Medical Image Computing and Computer Assisted Intervention:Proceedings of the 20th International Conference.Quebec City:Springer,2017:300-308. |
| 7 | BIGALKE A, HANSEN L, MOK T C W,et al .Unsupervised 3D registration through optimization-guided cyclical self-training[C]∥ GREENSPAN H,MADABHUSHI A,MOUSAVI P,et al.Medical Image Computing and Computer Assisted Intervention:Proceedings of the 26th International Conference.Vancouver:Springer,2023:677-687. |
| 8 | ZHANG J .Inverse-consistent deep networks for unsupervised deformable image registration[EB/OL].(2018-09-10)[2023-08-27].. |
| 9 | BALAKRISHNAN G, ZHAO A, SABUNCU M R,et al .Voxelmorph:a learning framework for deformable medical image registration[J].IEEE Transactions on Medical Imaging,2019,38(8):1788-1800. |
| 10 | 彭昆,张桂梅,王杰,等 .基于可变形卷积和多尺度特征聚焦的X线图像非刚性配准[J].生物医学工程学杂志,2023,40(3):492-498,507. |
| PENG Kun, ZHANG Guimei, WANG Jie,et al .Non-rigid registration for medical images based on deformable convolution and multi-scale feature focusing modules[J].Journal of Biomedical Engineering,2023,40(3):492-498,507. | |
| 11 | HERING A, VAN GINNEKEN B, HELDMANN S .mlVIRNet:multilevel variational image registration network[C]∥ SHEN D,LIU T,Peters T M,et al.Medical Image Computing and Computer Assisted Intervention:Proceedings of the 22nd International Conference.Shenzhen:Springer,2019:257-265. |
| 12 | ROHé M M, DATAR M, HEIMANN T,et al .SVF-Net:learning deformable image registration using shape matching[C]∥ DESCOTEAUX M,MAIER-HEIN L,FRANZ A,et al.Medical Image Computing and Computer Assisted Intervention:Proceedings of the 20th International Conference.Quebec City:Springer,2017:266-274. |
| 13 | CHEN J, LI Y, DU Y,et al .Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks[J].Medical Physics,2020,47(12):6366-6380. |
| 14 | HEINRICH M P, JENKINSON M, BRADY M,et al .MRF-based deformable registration and ventilation estimation of lung CT[J].IEEE Transactions on Medical Imaging,2013,32(7):1239-1248. |
| 15 | JIANG Z, YIN F F, GE Y,et al .A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration[J].Physics in Medicine & Biology,2020,65(1):015011/1-13. |
| 16 | DE VOS B D, BERENDSEN F F, VIERGEVER M A,et al .A deep learning framework for unsupervised affine and deformable image registration[J].Medical Image Analysis,2019,52:128-143. |
| 17 | YANG J, YANG J H, ZHAO F,et al .An unsupervised multi-scale framework with attention-based network (MANet) for lung 4D-CT registration[J].Physics in Medicine & Biology,2021,66(13):135008/1-13. |
| 18 | HO T T, KIM W J, LEE C H,et al .An unsupervised image registration method employing chest computed tomography images and deep neural networks[J].Computers in Biology and Medicine,2023,154:106612/1-11. |
| 19 | JADERBERG M, SIMONYAN K, ZISSERMAN A.Spatial transformer networks[C]∥ Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge:MIT Press,2015:2017-2025. |
| 20 | JIA X, THORLEY A, CHEN W,et al .Learning a model-driven variational network for deformable image registration[J].IEEE Transactions on Medical Imaging,2021,41(1):199-212. |
| 21 | MOK T C W, CHUNG A C S C .Fast symmetric diffeomorphic image registration with convolutional neural networks[C]∥ Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:4643-4652. |
| 22 | 石磊,籍庆余,陈清威,等 .视觉Transformer在医学图像分析中的应用研究综述[J].计算机工程与应用,2023,59(8):41-55. |
| SHI Lei, JI Qingyu, CHEN Qingwei,et al .Review of research on application of vision transformer in medical image analysis[J].Computer Engineering and Applications,2023,59(8):41-55. | |
| 23 | DEVLIN J, CHANG M W, LEE K,et al .BERT:pre-training of deep bidirectional transformers for language understanding[C]∥ Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Minneapolis:Association for Computational Linguistics,2019:4171-4186. |
| 24 | CASTILLO E, CASTILLO R, MARTINEZ J,et al .Four-dimensional deformable image registration using trajectory modeling[J].Physics in Medicine & Biology,2009,55(1):305-327. |
| 25 | CASTILLO R, CASTILLO E, GUERRA R,et al .A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets[J].Physics in Medicine & Biology,2009,54(7):1849-1870. |
/
| 〈 |
|
〉 |