收稿日期: 2022-09-02
网络出版日期: 2022-12-05
基金资助
安徽省重点研发计划项目(2022k07020006);安徽省高校自然科学研究重大项目(KJ2021ZD0004);安徽省自然科学基金资助项目(2108085MF232);公安部重点实验室开放课题(2017FMKFKT08)
Cross-Domain Pressure Footprint Images Retrieval Based on Mutual Information Disentangled Representations
Received date: 2022-09-02
Online published: 2022-12-05
Supported by
the Key R&D Program of Anhui Province(2022k07020006);the University Natural Science Research Major Program of Anhui Province(KJ2021ZD0004);the Natural Science Foundation of Anhui Province(2108085MF232)
足迹作为人体生物特征之一,在生物识别领域具有重要意义,而同一对象的不同鞋型压力足迹图像在足迹轮廓特征上具有显著性差异,导致其类内差异大。针对压力足迹图像的跨域检索,文中提出了一种基于互信息解耦表示的跨域压力足迹图像检索方法。首先,构建了一个包含200人足迹图像的多域压力足迹数据集,从定性和定量两个角度分析跨域压力足迹图像的特点;其次,采用两个独立的编码器实现图像解耦模块,该模块将压力足迹图像解耦为域特定表示和域共享表示,通过域分类法保证域特定表示包含更多域相关的信息;然后,通过最小化互信息损失扩大域特定表示和域共享表示之间的距离,同时,为避免解耦过程中信息的丢失,基于域特定表示和域共享表示重构原始压力足迹图像;最后,通过特征提取模块进一步提取域共享表示的深层卷积特征,经过度量模块计算不同特征间的关联度,从而实现跨域压力足迹图像检索。对比及消融实验结果表明,该方法的解耦模块具有一定的有效性,在多域压力足迹数据集上的性能表现良好,首位查询结果的检索准确率达到79.83%,平均准确率达到65.48%。
张艳, 许昌康, 曹丽青, 等 . 基于互信息解耦表示的跨域压力足迹图像检索[J]. 华南理工大学学报(自然科学版), 2023 , 51(5) : 78 -85 . DOI: 10.12141/j.issn.1000-565X.220572
As one of human biometric features, footprint is of great significance in the field of biometric identification. However, the pressure footprint images of different shoe types for the same person have significant differences in the footprint contour features, leading to large intra-class differences. For cross-domain retrieval of pressure footprint images, this paper proposed a cross-domain pressure footprint images retrieval method based on mutual information disentangled representations. Firstly, a multi-domain pressure footprint dataset containing 200 people’s footprint images was constructed and the characteristics of cross-domain pressure footprint images were analyzed from qualitative and quantitative perspectives. Secondly, two independent encoders were used to construct an image disentanglement module, which disentangles the pressure footprint images into a domain-specific representation and a domain-shared representation, and ensures that the domain-specific representation contains more domain-related information through domain classification. Then, the distance between the domain-specific representation and the domain-shared representation was enlarged by minimizing mutual information loss. At the same time, in order to avoid the loss of information in the disentangled process, the original pressure footprint image was reconstructed based on the domain-specific representation and the domain-shared representation. Finally, the deep convolution features of the domain-shared representation were further extracted by feature extraction module and the cross-domain pressure footprint images retrieval was realized through the metric module which calculates the correlation degree between different features. The results of comparison and ablation experiments show that the disentanglement module of this method is effective and performs well on multi-domain pressure footprint dataset. The retrieval accuracy of the first query result reached 79.83%, and the average accuracy reached 65.48%.
| 1 | 金益锋,白艳平,刘寰 .全国16个省份足迹自动识别系统应用情况分析[J].刑事技术,2017,42(6):504-507. |
| JIN Yifeng, BAI Yanping, LIU Huan .Application analysis on shoeprint automatic identification system from China’s 16 provinces[J].Forensic Science and Technology,2017,42(6):504-507. | |
| 2 | 史力民,马建平 .足迹学[M].北京:中国人民公安大学出版社,2014:1-12. |
| 3 | KANCHAN T, MENEZES R G, MOUDGIL R,et al .Stature estimation from foot dimensions[J].Forensic Science International,2008,179(2/3):241.e1-241.e5. |
| 4 | KEATSAMARN T, PINTAVIROOJ C .Footprint identification using deep learning[C]∥ Proceedings of 2018 the 11th Biomedical Engineering International Conference.Chiang Mai:IEEE,2018:1-4. |
| 5 | 陈杨,曾诚,程成,等 .一种基于CNN的足迹图像检索与匹配方法[J].南京师范大学学报(工程技术版),2018,18(3):39-45. |
| CHEN Yang, ZENG Cheng, CHENG Cheng,et al .A CNN-based approach to footprint image retrieval and matching[J].Journal of Nanjing Normal University (Engineering and Technology Edition),2018,18(3):39-45. | |
| 6 | 鲍文霞,瞿金杰,王年,等 .基于空间聚合加权卷积神经网络的力触觉足迹识别[J].东南大学学报(自然科学版),2020,50(5):959-964. |
| BAO Wenxia, QU Jinjie, WANG Nian,et al .Force-tactile footprint recognition based on spatial aggregation weighted convolutional neural network[J].Journal of Southeast University (Natural Science Edition),2020,50(5):959-964. | |
| 7 | 张艳,吴洛天,王年,等 .基于多模块关系网络的2D足迹分类[J].华南理工大学学报(自然科学版),2021,49(6):66-76. |
| ZHANG Yan, WU Luotian, WANG Nian,et al .2D footprint classification based on multiple-module relation network[J].Journal of South China University of Technology (Natural Science Edition),2021,49(6):66-76. | |
| 8 | 鲍文霞,茅丽丽,王年,等 .基于注意力双分支网络的跨模态足迹检索[J].东南大学学报(自然科学版),2021,51(5):914-922. |
| BAO Wenxia, MAO Lili, WANG Nian,et al .Cross-modal footprint retrieval based on the two-branch CNN with attention[J].Journal of Southeast University (Natural Science Edition),2021,51(5):914-922. | |
| 9 | 李浩然,周小平,王佳 .跨域图像检索综述[J].计算机工程与应用,2022,58(15):18-36. |
| LI Haoran, ZHOU Xiaoping, WANG Jia .Review of cross-domain image retrieval[J].Computer Engineering and Applications,2022,58(15):18-36. | |
| 10 | LIU F C, GAO C Q, SUN Y Q,et al .Infrared and visible cross-modal image retrieval through shared features[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,31(11):4485-4496. |
| 11 | PAUL S, DUTTA T, BISWAS S .Universal cross-domain retrieval:generalizing across classes and domains[C]∥ Proceedings of 2021 IEEE/CVF International Conference on Computer Vision.Montreal:IEEE,2021:12036-12044. |
| 12 | YU Q, SONG J, SONG Y Z,et al .Fine-grained instance-level sketch-based image retrieval [J].International Journal of Computer Vision,2021,129(2):484-500. |
| 13 | CHEN Y D, ZHANG Z L, WANG Y F,et al .AE-Net:fine-grained sketch-based image retrieval via attention-enhanced network[J].Pattern Recognition,2022,122:108291/1-15. |
| 14 | LEE H Y, TSENG H Y, MAO Q,et al .DRIT++:diverse image-to-image translation via disentangled representations[J].International Journal of Computer Vision,2020,128:2402-2417. |
| 15 | WU A, HAN Y H, ZHU L,et al .Instance-invariant domain adaptive object detection via progressive disentanglement [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(8):4178-4193. |
| 16 | BENGIO Y, COURVILLE A, VINCENT P .Representation learning:a review and new perspectives[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):1798-1828. |
| 17 | BELGHAZI M I, BARATIN A, RAJESWAR S,et al .Mutual information neural estimation[C]∥ Proceedings of the 35th International Conference on Machine Learning.Stockholm:IMLS,2018:864-873. |
| 18 | PENG X, HUANG Z, ZHU Y,et al .Federated adversarial domain adaptation[C]∥ Proceedings of the Eighth International Conference on Learning Representations.Ethiopia:ICLR,2020:1-19. |
| 19 | HWANG H J, KIM G H, HONG S,et al .Variational interaction information maximization for cross-domain disentanglement[C]∥ Proceedings of the 34th Conference on Neural Information Processing Systems.Vancouver:NIPS Foundation,2020:1-26. |
/
| 〈 |
|
〉 |