计算机科学与技术

基于材料结构条件先验的高噪声STEM图像原子结构分割方法

  • 郭礼华 ,
  • 林延域 ,
  • 陈轲
展开
  • 1.华南理工大学 电子与信息学院,广东 广州 510640
    2.中国电信股份有限公司 广东分公司,广东 广州 510440
    3.鹏城实验室,广东 深圳 518055
郭礼华(1978—),男,博士,副教授,主要从事机器学习与图像理解研究。E-mail:guolihua@scut.edu.cn
郭礼华(1978—),男,博士,副教授,主要从事机器学习与图像理解研究。E-mail:guolihua@scut.edu.cn

收稿日期: 2025-01-23

  网络出版日期: 2025-05-06

基金资助

广东省自然科学基金项目(2023A1515011014)

An Atomic Structure Segmentation Method for High-Noise STEM Image Based on Materials Structural Priors

  • GUO Lihua ,
  • LIN Yanyu ,
  • CHEN Ke
Expand
  • 1.School of Electronics and Information Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China
    2.Guangdong Branch,China Telecom Co. ,Ltd. ,Guangzhou 510440,Guangdong,China
    3.Pengcheng Laboratory,Shenzhen 518055,Guangdong,China
郭礼华(1978—),男,博士,副教授,主要从事机器学习与图像理解研究。E-mail:guolihua@scut.edu.cn

Received date: 2025-01-23

  Online published: 2025-05-06

Supported by

the Natural Science Foundation of Guangdong Province(2023A1515011014)

摘要

扫描透射电子显微镜(STEM)可以在原子皮米级别上对物体材质进行电子成像,并利用图像进行原子结构解读。但是,要获得高质量的原子尺度STEM图像需要高端的STEM设备以及熟练的操作者,各种环境因素都会在STEM的成像过程中引入难以预计的非均匀噪声,从而严重影响图像质量,进而影响原子结构分析结果。基于深度神经网络的预测模型可以通过去噪或数据拟合来减少噪声的影响,但存在过拟合问题。该文将材料结构条件先验建模到深度神经网络模型中,设计了一种基于材料结构条件先验的高噪声STEM图像原子结构分割方法。该方法通过对比学习方式将材料结构条件先验建模成分割网络的注意力(包括自注意力和交叉注意力)并加以计算,不仅使得分割网络能够自适应地关注图像中的关键区域,还能自适应地关注来自结构坐标向量模态的控制信息。在仿真测试集中,该方法相比AtomAI Segmentor方法,在倒角距离、Jaccard分数和F1分数上分别提升175%、49.7%和42.7%;相比作者课题组早期提出的多尺度方法,在倒角距离、Jaccard分数和F1分数上分别提升167%、28%和23.9%。在实验室样本测试集中,该方法相比AtomAI Segmentor方法,在倒角距离、Jaccard分数和F1分数上分别提升63%、9.3%和7.4%;相比作者课题组早期提出的多尺度方法,在倒角距离上提升12.8%,在Jaccard分数和F1分数上性能持平。材料结构条件先验的引入,使得分割网络模型能更准确地分割高噪声STEM图像中的原子结构,并预测被噪声或顶层遮挡的次级结构信息。

本文引用格式

郭礼华 , 林延域 , 陈轲 . 基于材料结构条件先验的高噪声STEM图像原子结构分割方法[J]. 华南理工大学学报(自然科学版), 2025 , 53(11) : 27 -36 . DOI: 10.12141/j.issn.1000-565X.250032

Abstract

Scanning transmission electron microscopy (STEM) can perform electron imaging of material properties at the atomic picometer level and interpret the atomic structure using the obtained images. However, obtaining high-quality atomic-scale STEM images requires advanced STEM equipment and skilled operators. Various environmental factors can introduce unpredictable non-uniform noise during the STEM imaging process, thereby significantly affecting image quality and consequently influencing the results of atomic structure analysis. The prediction model based on deep neural networks can reduce the impact of noise through denoising or data fitting, but there exists a problem of overfitting. This paper introduces materials structure conditions as priors in the deep neural network model and designs a method for atomic structure segmentation of high-noise STEM images based on materials structural priors. In this method, the materials structural priors are modelled as the attention (including self-attention and cross-attention) of the segmentation network and are calculated, which not only enables the segmentation network to adaptively focus on the key regions of the image but also to adaptively focus on the control information from the structural coordinate vector modalities. In the simulation test set, as compared with AtomAI Segmentor method, the proposed method improves the chamfer distance, Jaccard and F1 metrics by 175%, 49.7% and 42.7%, respectively; as compared with the early multi-scale method proposed by the research group, it improves the chamfer distance, Jaccard and F1 metrics by 167%, 28% and 23.9%, respectively. In the laboratory sample test set, as compared with AtomAI Segmentor method, the proposed method improves the chamfer distance, Jaccard and F1 metrics by 63%, 9.3% and 7.4%, respectively; as compared with the early multi-scale method proposed by the research group, it improves the chamfer distance by 12.8%, and the Jaccard and F1 metrics remain largely unchanged. The introduction of materials structural priors enables the segmentation network model to more accurately segment the atomic structure in high-noise STEM images and predict the secondary structure information that is affected by noise or top-level occlusion.

参考文献

[1] BINEV P, BLANCO-SILVA F, BLOM D,et al .High-quality image formation by nonlocal means applied to high-angle annular dark-field scanning transmission electron microscopy (HAADF-STEM)[J].Springer20121(1):127-145.
[2] MARKS L .Wiener-filter enhancement of noisy HREM images[J].Ultramicroscopy199662(1):43-52.
[3] KUMAR S, KUMAR P, GUPTA M,et al .Performance comparison of median and Wiener filter in image de-noising[J].International Journal of Computer Applications201012(1):27-31.
[4] NORD M, VULLUM P E, MACLAREN I,et al .Atomap: a new software tool for the automated analysis of atomic resolution images using two-dimensional Gau-ssian fitting[J].Advanced structural and chemical imaging20173(1):1-12.
[5] OHNISHI I, MIURA H .Atomic resolution observations for beam-sensitive asbestos samples using low-dose STEM imaging techniques[C]∥Proceedings of the 13th Asia Pacific Microscopy Congress 2025.Brisbane:ACMM2 in Microscopy and Analysis Society,2025:160-166.
[6] DAN J, ZHANG C, ZHAO X,et al .Symmetry quantification and segmentation in STEM imaging through Zernike moments[J].Chinese Physics B202433(8):1056-1064.
[7] ZIATDINOV M, GHOSH A, WONG T,et al .AtomAI:a deep learning framework for analysis of image and spectroscopy data in (scanning) transmission electron microscopy and beyond[EB/OL].(2021-05-17) [2024-12-04]..
[8] GUO Y, KALININ S, CAI H,et al .Defect detection in atomic-resolution images via unsupervised learning with translational invariance[J].npj Computational Materials20217(1):180-190.
[9] YANG S, CHOI W, CHO B,et al .Deep learning-assisted quantification of atomic dopants and defects in 2D materials[J].Advanced Science20218(16):1-19.
[10] RAGONE M, YURKIV V, SONG B,et al .Atomic column heights detection in metallic nanoparticles using deep convolutional learning[J].Computational Materials Science2020180(1):1-16.
[11] ZHANG C, FENG J, LUIS R D,et al .Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks[J].Ultramicroscopy2020210(10):1-17.
[12] CHENG D, SHA W, GUO Y,et al .A real-time deep learning model to narrow the gap between atomic scanning transmission electron microscopy and theory calculations: recognition, reconstruction, and simulation[J].MRS Bulletin202550(2):101-114.
[13] CABRERA C, CERVANTES D, MU?OZ F,et al .Classifying nanostructured and heterogeneous materials from transmission electron microscopy images using convolutional neural networks[J].Neural Computing & Applications202234(1):11035-11047.
[14] ZHU L, TANG J, LI B,et al .Artificial neuron networks enabled identification and characterizations of 2D materials and van der Waals heterostructures[J].ACS Nano202216(2):2721-2729.
[15] 南虎,麻晓晶,赵海博,等 .基于YOLOv3框架的高分辨电镜图像原子峰位置检测[J].物理学报202170(7):076803/1-8.
  Hu NAN, MA Xiao-jing, ZHAO Hai-bo,et al .Atomic peak location detection in high-resolution electron microscopy images based on YOLOv3[J].Acta Physica Sinica202170(7):076803/1-8.
[16] CHU T, ZHOU L, ZHANG B,et al .Accurate atomic scanning transmission electron microscopy analysis enabled by deep learning[J].Nano Research202417(7),2971-2980.
[17] LEE C, KHAN A, LUO D,et al .Deep learning ena-bled strain mapping of single-atom defects in two-dimensional transition metal dichalcogenides with sub-picometer precision[J].Nano Letters202020(5):3369-3377.
[18] JACOB M, LIU P, JENS K,et al .A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images[J].Advan-ced Theory Simulations20181(1):1-17.
[19] LIN R, ZHANG R, WANG C,et al .TEMImageNet training library and AtomSegNet deep-learning models for high-precision atom segmentation,localization,denoising,and deblurring of atomic-resolution images[J].Scientific Reports202111(2):1-18.
[20] LIN Y, YAN Z, TSANG C S,et al .A multiscale deep-learning model for atom identification from low-signal-to-noise-ratio transmission electron microscopy images[J].Small Science20233(8):1-15.
[21] DOSOVITSKIY A A,BEYER,LU, KOLESNIKOV A,et al .An image is worth 16×16 words: transformers for image recognition at scale[EB/OL].(2021-06-03) [2024-12-04]..
[22] MADSEN J, SUSI T .The abTEM code: transmission electron microscopy from first principles[J].Open Research Europe202115(2):1-24.
[23] RONNEBERGER O, FISCHER P, BROX T .U-Net: convolutional networks for biomedical image segmentation[EB/OL].(2015-05-18)[2024-12-04]..
文章导航

/