基于多目近红外视觉的多目标实时跟踪方法

doi:10.12141/j.issn.1000-565X.240427

摘要/Abstract

摘要：

近红外光学跟踪系统能够根据附着于被跟踪物体上的反光标记球实时还原被跟踪物体的运动，目前已被广泛应用于多种领域。该研究提出了一种对目标丢失具有一定鲁棒性的多目近红外目标实时跟踪方法。首先，针对反光标记球在近红外相机中的成像特性，利用灰度质心法提取各个反光标记球的几何中心，然后在各单目相机中使用SORT算法作为多目标跟踪方法对各个标记点进行帧间匹配，并根据对极几何原理，结合带权二分图匹配方法确定反光标记球在各个相机中像点的匹配关系，依据三角测量方法实时计算各个受跟踪反光标记球的三维空间坐标；其次，根据运动过程中各反光标记球之间的空间位置关系对反光标记球进行分组，识别属于同一物体的反光标记球，并根据同组反光标记球间的欧氏距离建立被跟踪物体与反光标记球的外观特征向量，以此作为物体丢失重现的匹配依据，而完全丢失后再重现的被跟踪物体利用外观特征向量的余弦距离进行重匹配；最后，对所提方法进行实验验证。实验结果表明：所提方法在不小于60 f/s的帧率下的跟踪精度约可达0.5 mm；另外，其可以对丢失的重现物体以及反光标记球进行正确的重匹配。

关键词: 多目视觉, 近红外光学跟踪, 立体匹配, 反光标记球

Abstract:

Near-infrared optical tracking systems can restore the movement of tracked objects in real time based on the markers attached to the tracked objects. This technology has now been widely adopted across numerous fields. This paper proposed a real-time tracking method for muli-objects that is robust to target loss. First, based on the imaging characteristics of reflective marker balls in near-infrared cameras, the geometric center of each marker was extracted using the grayscale centroid method. Then, the SORT algorithm was used as a multi-objetcs tracking method in each monocular camera to match each marker point between frames. The matching relationship of the image points of the markers in each camera was determined based on the principle of epipolar geometry combined with the weighted bipartite graph matching method, and the three-dimensional spatial coordinates of each tracked marker were calculated in real time based on the triangulation method. Next, the markers were grouped based on their spatial relationships during motion to identify markers belonging to the same object. Spatial feature vectors were established for tracked objects using the Euclidean distances between markers within the same group, serving as matching references for reappearing lost objects. When a fully lost object reproduced, re-matching is performed using cosine distance of these feature vectors. Finally, the proposed algorithm was experimentally verified. The experiment shows that the tracking accuracy of the proposed algorithm can reach about 0.5 mm at a speed of not less than 60 f/s. In addition, the lost reproduced objects and markers can be correctly re-matched.

Key words: multi-view vision, near-infrared optical tracking, stereo matching, marker

中图分类号:

TP391.41

陈忠, 王傲辰, 高心怡, 何利辉, 张宪民. 基于多目近红外视觉的多目标实时跟踪方法[J]. 华南理工大学学报(自然科学版), 2025, 53(7): 31-38.

CHEN Zhong, WANG Aochen, GAO Xinyi, HE Lihui, ZHANG Xianmin. Multi-Object Real-Time Tracking Method Based on Multi-View Near-Infrared Vision[J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(7): 31-38.

图/表 17

图1

图2

图3

图4

图5

图6

表1

图7

表2

相机外参"

序号	旋转矩阵	平移矩阵/mm
1	$100010001$	$000$
2	$0.365 9 0.731 0 - 0.575 9 - 0.684 5 0.630 7 0.365 6 0.630 4 0.260 5 0.731 2$	$1 181.417 7 - 629.616 4 766.573 7$
3	$- 0.838 5 0.460 4 - 0.291 6 - 0.407 7 - 0.174 9 0.896 2 0.361 6 0.870 3 0.334 3$	$649.287 3 - 1 898.638 8 1 477.047 8$

表2

图8

图9

图10

表3

表4

表5

图11

表6

获取的物体及反光标记球的特征"

序号	外观特征向量
序号	物体	反光标记球1	反光标记球2	反光标记球3
1a	$[41.77 64.49 93.84] T$	$[41.77 64.49] T$	$[41.77 93.84] T$	$[64.49 93.84] T$
1b	$[41.78 64.67 93.56] T$	$[41.78 64.67] T$	$[41.78 93.56] T$	$[64.67 93.56] T$
2a	$[35.26 60.38 64.11] T$	$[35.26 64.11] T$	$[35.26 60.38] T$	$[60.38 64.11] T$
2b	$[35.26 60.86 64.40] T$	$[35.26 64.40] T$	$[35.26 60.86] T$	$[60.86 64.40] T$
3	$[49.02 68.90 100.40] T$	$[49.02 68.90] T$	$[49.02 100.40] T$	$[68.90 100.40] T$
4	$[51.38 84.24 121.84] T$	$[51.38 84.24] T$	$[51.38 121.84] T$	$[84.24 121.84] T$

表6

参考文献 16

[1]	LOWE D G ．Distinctive image features from scale-invariant keypoints［J］．International Journal of Computer Vision，2004，60：91-110.
[2]	BAY H，ESS A， TUYTELAARS T，et al ．Speeded-up robust features （SURF）［J］．Computer Vision and Image Understanding，2008，110（3）：346-359.
[3]	ZHOU Z， WU B， DUAN J，et al ．Optical surgical instrument tracking system based on the principle of stereo vision［J］．Journal of Biomedical Optics，2017，22（6）：065005/1-15.
[4]	LIN Q， YANG R， ZHANG Z，et al ．Robust stereo-match algorithm for infrared markers in image-guided optical tracking system［J］．IEEE Access，2018，6：52421-52433.
[5]	王杰，仲重亮，朱伟东．一种用于光学器械跟踪的近红外双目系统［J］．红外与激光工程，2022，51（6）：161-170.
	WANG Jie， ZHONG Chongliang， ZHU Weidong ．A near-infrared binocular system for optical instrument tracking［J］．Infrared and Laser Engineering，2022，51（6）：161-170.
[6]	ZHANG M， WU B， YE C，et al ．Multiple instruments motion trajectory tracking in optical surgical navigation［J］．Optics Express，2019，27（11）：15827-15845.
[7]	ARISTIDOU A， LASENBY J ．Real-time marker prediction and CoR estimation in optical motion capture［J］．The Visual Computer，2013，29：7-26.
[8]	PIAZZA T， LUNDSTRÖM J， KUNZ A，et al ．Predicting missing markers in real-time optical motion capture［C］∥Proceedings of the Modelling the Physiological Human（3DPH2009）．Zermatt：Springer，2009：125-136.
[9]	BURKE M， LASENBY J ．Estimating missing marker positions using low dimensional Kalman smoothing［J］．Journal of Biomechanics，2016，49（9）：1854-1858.
[10]	BEWLEY A， GE Z，OTT L，et al ．Simple online and realtime tracking［C］∥Proceedings of the 2016 IEEE International Conference on Image Processing （ICIP）．Phoenix：IEEE，2016：3464-3468.
[11]	Ｒ HARTLEY， ZISSERMAN A，Multiple view geo-metry in computer vision［M］．2nd ed．Cambridge：Cambridge University Press，2003：239-242.
[12]	ZHANG Z ．A flexible new technique for camera calibration［J］．IEEE Transactions on Pattern Analysis and Machine Intelligence，2000，22（11）：1330-1334.
[13]	李斌，谭光华，高春鸣．红外摄像机组的一维标定物设计和标定［J］．计算机应用，2013，33（5）：1423-1427，1449.
	LI Bin， TAN Guanghua， GAO Chunming ．One-dimensional objects design and calibration in infrared camera group［J］．Journal of Computer Applications，2013，33（5）：1423-1427，1449.
[14]	KÜMMERLE R， GRISETTI G， STRASDAT H，et al ．g2o：a general framework for graph optimization［C］∥Proceedings of the 2011 IEEE International Conference on Robotics and Automation．Shanghai：IEEE，2011：3607-3613.
[15]	CHADDA A， ZHU W， LEU M C，et al ．Design implementation，and evaluation of optical low-cost motion capture system［C］∥Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference．Washington：ASME，2011：1451-1461.
[16]	PINTARIC T， KAUFMANN H ．Affordable infrared-optical pose-tracking for virtual and augmented reality［C］∥Proceedings of the Trends and Issues in Tracking for Virtual Environments Workshop，IEEE VR． Charlotte：IEEE，2007：44-51.

序号	f_X	f_Y	u	v
1	2 953.86	2 953.52	1 030.43	541.31
2	2 955.89	2 954.18	1 017.04	550.31
3	2 943.39	2 942.87	1 036.95	517.87

维度	平均绝对误差/mm	均方根误差/mm	最大误差/mm
X	0.110 0	0.142 0	0.320 9
Y	0.137 6	0.148 6	0.259 5
Z	0.141 6	0.166 6	0.392 2

方法	物距/m	真实值/mm	平均绝对误差/mm	均方根误差/mm	最大误差/mm
文献［5］	1.8	100	0.461 9	0.576 5	1.065 4
文献［15］	2.4	200	0.65		2.8
文献［16］	3	400		5.2	15
文中	1.8	502.63	0.174 5	0.135 8	0.449 5