Journal of South China University of Technology(Natural Science Edition) ›› 2022, Vol. 50 ›› Issue (6): 10-18.doi: 10.12141/j.issn.1000-565X.210603

Special Issue: 2022年计算机科学与技术

• Computer Science & Technology • Previous Articles     Next Articles

Image tampering localization based on mutil-scale transformer

LU Lu  ZHONG Wenyu  WU Xiaokun   

  1. School of Computer Science and Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China
  • Received:2021-09-17 Revised:2021-10-27 Online:2022-06-25 Published:2021-11-08
  • Contact: 吴小坤 (1980-),女,教授,主要从事数据分析和信息可视化研究 E-mail:wuxiaokun@ scut. edu. cn
  • About author:陆璐 (1971-),男,教授,主要从事计算机视觉和软件质量保障研究
  • Supported by:
    the National Social Science Foundation Key Project of China;the Major Program of the Zhongshan Industry-Academia-Research Fund

Abstract: With the continuous development of digital image processing technology, image tampering is no longer limited to a single method such as image splicing. The traces of malicious tampering are hidden through the post-processing of the image editing software, which leads to poor results of traditional image forgery detection algorithms and the tampering localization methods based on deep learning. Aiming at the problem of low accuracy of existing image tampering algorithms, an end-to-end image tampering location network based on Multi-Scale Visual Transformer is proposed. The network combines a transformer and a convolutional encoder to extract the feature difference between the tampered area and the non-tampered area. Multi-Scale Transformer models the spatial information of image block sequences of different sizes, so that the network can adapt to tampered areas of various shapes and sizes. Experimental results show that the F1 and AUC scores of the proposed algorithm in the CASIA and NIST2016 test sets are 0.431、0.877、0.728 and 0.971, respectively, which are significantly improved co- mpared to the existing mainstream algorithms. Moreover, the algorithm proposed in this paper is robust against JPEG compression attacks.

Key words: Deep learning, visual Transformer, image tampering, vertical and horizontal attention

CLC Number: