Journal of South China University of Technology(Natural Science) >

2024 , Vol. 52 >Issue 10: 135 - 145

DOI: https://doi.org/10.12141/j.issn.1000-565X.230726

Image Processing

Algorithm for Multiscale Residual Deformable Lung CT Image Registration

  • LIU Weipeng ,
  • LI Xu ,
  • REN Ziwen ,
  • QI Yedong
Expand
  • 1.School of Artificial Intelligence，Hebei University of Technology，Tianjin 300401，China
    2.School of Health Sciences and Biomedical Engineering，Hebei University of Technology，Tianjin 300401，China
    3.Institute of Biomedical Engineering，Chinese Academy of Medical Sciences and Peking Union Medical College，Tianjin 300192，China

Received date: 2023-11-22

  Online published: 2024-01-31

Supported by

the National Key R & D Program of China(2020YFB1313703);the National Natural Science Foundation of China(62027813);the Key R & D Program of Hebei Province(21372003D);the Natural Science Foundation of Hebei Province(F2022202054)

Fold

Abstract

The 4-dimensional CT (4D-CT) images of the lungs undergo large deformations due to respiration and heartbeat, and the scale of motion within the lungs may be larger than the structures of interest (blood vessels, airways, etc.) that the algorithm uses for the optimization process, which may result in the registration algorithms only aligning the obvious features such as blood vessels and airways. To address the problem of high variability of the aligned intensities for structures with large deformations such as the lung parenchyma contour, this paper proposed a multi-scale residual deformable lung CT image alignment algorithm framework based on unsupervised end-to-end deep learning. A multi-scale deep residual network in the form of an encoder-decoder structure was used as a generative model for the deformation field in the proposed registration framework, so as to enhance the feature representation, to increase the effective parameter utilization efficiency parameters and effectively improve the convergence ability of the network. A multi-resolution self-attentive fusion module was used to improve the network’s ability to perceive multi-scale information. And a hopping connection containing a feature correction extraction module was designed to selectively extract the feature maps output by the encoder and recalibrate them for the decoder to learn the alignment offsets. Finally, this paper compared the proposed alignment algorithm with traditional algorithms and the current state-of-the-art unsupervised alignment algorithms on the Dir-lab public dataset. The results show that, the target alignment error of the proposed registration algorithm framework on the Dir-lab public dataset can reach 1.44 mm ± 1.24 mm, which is better than traditional algorithms and the mainstream unsupervised alignment algorithm. In addition, the estimation of the dense deformation vector field takes less than 2.00 s with the control folding voxel less than 0.1%, indicating the great potential of the algorithm in studying time-sensitive lungs.

Key words： deep learning; lung CT image; image registration; unsupervised learning

Cite this article

LIU Weipeng , LI Xu , REN Ziwen , QI Yedong . Algorithm for Multiscale Residual Deformable Lung CT Image Registration[J]. Journal of South China University of Technology(Natural Science), 2024 , 52(10) : 135 -145 . DOI: 10.12141/j.issn.1000-565X.230726

References

1 AVANTS B B， TUSTISON N J， SONG G，et al ．A reproducible evaluation of ANTs similarity metric performance in brain image registration［J］．Neuroimage，2011，54（3）：2033-2044.
2 AVANTS B B， EPSTEIN C L， GROSSMAN M，et al ．Symmetric diffeomorphic image registration with cross-correlation：evaluating automated labeling of elderly and neurodegenerative brain［J］．Medical Image Analysis，2008，12（1）：26-41.
3 BEG M F， MILLER M I， TROUVé A，et al ．Computing large deformation metric mappings via geodesic flows of diffeomorphisms［J］．International Journal of Computer Vision，2005，61：139-157.
4 FU Y， LEI Y， WANG T，et al ．Deep learning in medical image registration：a review［J］．Physics in Medicine & Biology，2020，65（20）：20TR01/1-32.
5 CHEE E， WU Z ．AIRNet：self-supervised affine registration for 3D medical images using neural networks［EB/OL］．（2018-10-15）［2023-08-27］．.
6 CAO X， YANG J， ZHANG J，et al ．Deformable image registration based on similarity-steered CNN regression［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：300-308.
7 BIGALKE A， HANSEN L， MOK T C W，et al ．Unsupervised 3D registration through optimization-guided cyclical self-training［C］∥ GREENSPAN H，MADABHUSHI A，MOUSAVI P，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 26th International Conference．Vancouver：Springer，2023：677-687.
8 ZHANG J ．Inverse-consistent deep networks for unsupervised deformable image registration［EB/OL］．（2018-09-10）［2023-08-27］．.
9 BALAKRISHNAN G， ZHAO A， SABUNCU M R，et al ．Voxelmorph：a learning framework for deformable medical image registration［J］．IEEE Transactions on Medical Imaging，2019，38（8）：1788-1800.
10 彭昆，张桂梅，王杰，等 ．基于可变形卷积和多尺度特征聚焦的X线图像非刚性配准［J］．生物医学工程学杂志，2023，40（3）：492-498，507.
  PENG Kun， ZHANG Guimei， WANG Jie，et al ．Non-rigid registration for medical images based on deformable convolution and multi-scale feature focusing modules［J］．Journal of Biomedical Engineering，2023，40（3）：492-498，507.
11 HERING A， VAN GINNEKEN B， HELDMANN S ．mlVIRNet：multilevel variational image registration network［C］∥ SHEN D，LIU T，Peters T M，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 22nd International Conference．Shenzhen：Springer，2019：257-265.
12 ROHé M M， DATAR M， HEIMANN T，et al ．SVF-Net：learning deformable image registration using shape matching［C］∥ DESCOTEAUX M，MAIER-HEIN L，FRANZ A，et al．Medical Image Computing and Computer Assisted Intervention：Proceedings of the 20th International Conference．Quebec City：Springer，2017：266-274.
13 CHEN J， LI Y， DU Y，et al ．Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks［J］．Medical Physics，2020，47（12）：6366-6380.
14 HEINRICH M P， JENKINSON M， BRADY M，et al ．MRF-based deformable registration and ventilation estimation of lung CT［J］．IEEE Transactions on Medical Imaging，2013，32（7）：1239-1248.
15 JIANG Z， YIN F F， GE Y，et al ．A multi-scale framework with unsupervised joint training of convolutional neural networks for pulmonary deformable image registration［J］．Physics in Medicine & Biology，2020，65（1）：015011/1-13.
16 DE VOS B D， BERENDSEN F F， VIERGEVER M A，et al ．A deep learning framework for unsupervised affine and deformable image registration［J］．Medical Image Analysis，2019，52：128-143.
17 YANG J， YANG J H， ZHAO F，et al ．An unsupervised multi-scale framework with attention-based network （MANet） for lung 4D-CT registration［J］．Physics in Medicine & Biology，2021，66（13）：135008/1-13.
18 HO T T， KIM W J， LEE C H，et al ．An unsupervised image registration method employing chest computed tomography images and deep neural networks［J］．Computers in Biology and Medicine，2023，154：106612/1-11.
19 JADERBERG M， SIMONYAN K， ZISSERMAN A.Spatial transformer networks［C］∥ Proceedings of the 28th International Conference on Neural Information Processing Systems．Cambridge：MIT Press，2015：2017-2025.
20 JIA X， THORLEY A， CHEN W，et al ．Learning a model-driven variational network for deformable image registration［J］．IEEE Transactions on Medical Imaging，2021，41（1）：199-212.
21 MOK T C W， CHUNG A C S C ．Fast symmetric diffeomorphic image registration with convolutional neural networks［C］∥ Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition．Seattle：IEEE，2020：4643-4652.
22 石磊，籍庆余，陈清威，等 ．视觉Transformer在医学图像分析中的应用研究综述［J］．计算机工程与应用，2023，59（8）：41-55.
  SHI Lei， JI Qingyu， CHEN Qingwei，et al ．Review of research on application of vision transformer in medical image analysis［J］．Computer Engineering and Applications，2023，59（8）：41-55.
23 DEVLIN J， CHANG M W， LEE K，et al ．BERT：pre-training of deep bidirectional transformers for language understanding［C］∥ Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies．Minneapolis：Association for Computational Linguistics，2019：4171-4186.
24 CASTILLO E， CASTILLO R， MARTINEZ J，et al ．Four-dimensional deformable image registration using trajectory modeling［J］．Physics in Medicine & Biology，2009，55（1）：305-327.
25 CASTILLO R， CASTILLO E， GUERRA R，et al ．A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets［J］．Physics in Medicine & Biology，2009，54（7）：1849-1870.
Options
Outlines

/

〈 〉