Multi-Object Real-Time Tracking Method Based on Multi-View Near-Infrared Vision

doi:10.12141/j.issn.1000-565X.240427

Abstract

Abstract:

Near-infrared optical tracking systems can restore the movement of tracked objects in real time based on the markers attached to the tracked objects. This technology has now been widely adopted across numerous fields. This paper proposed a real-time tracking method for muli-objects that is robust to target loss. First, based on the imaging characteristics of reflective marker balls in near-infrared cameras, the geometric center of each marker was extracted using the grayscale centroid method. Then, the SORT algorithm was used as a multi-objetcs tracking method in each monocular camera to match each marker point between frames. The matching relationship of the image points of the markers in each camera was determined based on the principle of epipolar geometry combined with the weighted bipartite graph matching method, and the three-dimensional spatial coordinates of each tracked marker were calculated in real time based on the triangulation method. Next, the markers were grouped based on their spatial relationships during motion to identify markers belonging to the same object. Spatial feature vectors were established for tracked objects using the Euclidean distances between markers within the same group, serving as matching references for reappearing lost objects. When a fully lost object reproduced, re-matching is performed using cosine distance of these feature vectors. Finally, the proposed algorithm was experimentally verified. The experiment shows that the tracking accuracy of the proposed algorithm can reach about 0.5 mm at a speed of not less than 60 f/s. In addition, the lost reproduced objects and markers can be correctly re-matched.

Key words: multi-view vision, near-infrared optical tracking, stereo matching, marker

CLC Number:

TP391.41

CHEN Zhong, WANG Aochen, GAO Xinyi, HE Lihui, ZHANG Xianmin. Multi-Object Real-Time Tracking Method Based on Multi-View Near-Infrared Vision[J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(7): 31-38.

Figures/Tables 17

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Fig.7

Table 2

Camera extrinsic parameters"

序号	旋转矩阵	平移矩阵/mm
1	$100010001$	$000$
2	$0.365 9 0.731 0 - 0.575 9 - 0.684 5 0.630 7 0.365 6 0.630 4 0.260 5 0.731 2$	$1 181.417 7 - 629.616 4 766.573 7$
3	$- 0.838 5 0.460 4 - 0.291 6 - 0.407 7 - 0.174 9 0.896 2 0.361 6 0.870 3 0.334 3$	$649.287 3 - 1 898.638 8 1 477.047 8$

Table 2

Fig.8

Fig.9

Fig.10

Table 3

Table 4

Table 5

Fig.11

Table 6

Features of tracked objects and markers"

序号	外观特征向量
序号	物体	反光标记球1	反光标记球2	反光标记球3
1a	$[41.77 64.49 93.84] T$	$[41.77 64.49] T$	$[41.77 93.84] T$	$[64.49 93.84] T$
1b	$[41.78 64.67 93.56] T$	$[41.78 64.67] T$	$[41.78 93.56] T$	$[64.67 93.56] T$
2a	$[35.26 60.38 64.11] T$	$[35.26 64.11] T$	$[35.26 60.38] T$	$[60.38 64.11] T$
2b	$[35.26 60.86 64.40] T$	$[35.26 64.40] T$	$[35.26 60.86] T$	$[60.86 64.40] T$
3	$[49.02 68.90 100.40] T$	$[49.02 68.90] T$	$[49.02 100.40] T$	$[68.90 100.40] T$
4	$[51.38 84.24 121.84] T$	$[51.38 84.24] T$	$[51.38 121.84] T$	$[84.24 121.84] T$

Table 6

References 16

[1]	LOWE D G ．Distinctive image features from scale-invariant keypoints［J］．International Journal of Computer Vision，2004，60：91-110.
[2]	BAY H，ESS A， TUYTELAARS T，et al ．Speeded-up robust features （SURF）［J］．Computer Vision and Image Understanding，2008，110（3）：346-359.
[3]	ZHOU Z， WU B， DUAN J，et al ．Optical surgical instrument tracking system based on the principle of stereo vision［J］．Journal of Biomedical Optics，2017，22（6）：065005/1-15.
[4]	LIN Q， YANG R， ZHANG Z，et al ．Robust stereo-match algorithm for infrared markers in image-guided optical tracking system［J］．IEEE Access，2018，6：52421-52433.
[5]	王杰，仲重亮，朱伟东．一种用于光学器械跟踪的近红外双目系统［J］．红外与激光工程，2022，51（6）：161-170.
	WANG Jie， ZHONG Chongliang， ZHU Weidong ．A near-infrared binocular system for optical instrument tracking［J］．Infrared and Laser Engineering，2022，51（6）：161-170.
[6]	ZHANG M， WU B， YE C，et al ．Multiple instruments motion trajectory tracking in optical surgical navigation［J］．Optics Express，2019，27（11）：15827-15845.
[7]	ARISTIDOU A， LASENBY J ．Real-time marker prediction and CoR estimation in optical motion capture［J］．The Visual Computer，2013，29：7-26.
[8]	PIAZZA T， LUNDSTRÖM J， KUNZ A，et al ．Predicting missing markers in real-time optical motion capture［C］∥Proceedings of the Modelling the Physiological Human（3DPH2009）．Zermatt：Springer，2009：125-136.
[9]	BURKE M， LASENBY J ．Estimating missing marker positions using low dimensional Kalman smoothing［J］．Journal of Biomechanics，2016，49（9）：1854-1858.
[10]	BEWLEY A， GE Z，OTT L，et al ．Simple online and realtime tracking［C］∥Proceedings of the 2016 IEEE International Conference on Image Processing （ICIP）．Phoenix：IEEE，2016：3464-3468.
[11]	Ｒ HARTLEY， ZISSERMAN A，Multiple view geo-metry in computer vision［M］．2nd ed．Cambridge：Cambridge University Press，2003：239-242.
[12]	ZHANG Z ．A flexible new technique for camera calibration［J］．IEEE Transactions on Pattern Analysis and Machine Intelligence，2000，22（11）：1330-1334.
[13]	李斌，谭光华，高春鸣．红外摄像机组的一维标定物设计和标定［J］．计算机应用，2013，33（5）：1423-1427，1449.
	LI Bin， TAN Guanghua， GAO Chunming ．One-dimensional objects design and calibration in infrared camera group［J］．Journal of Computer Applications，2013，33（5）：1423-1427，1449.
[14]	KÜMMERLE R， GRISETTI G， STRASDAT H，et al ．g2o：a general framework for graph optimization［C］∥Proceedings of the 2011 IEEE International Conference on Robotics and Automation．Shanghai：IEEE，2011：3607-3613.
[15]	CHADDA A， ZHU W， LEU M C，et al ．Design implementation，and evaluation of optical low-cost motion capture system［C］∥Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference．Washington：ASME，2011：1451-1461.
[16]	PINTARIC T， KAUFMANN H ．Affordable infrared-optical pose-tracking for virtual and augmented reality［C］∥Proceedings of the Trends and Issues in Tracking for Virtual Environments Workshop，IEEE VR． Charlotte：IEEE，2007：44-51.

序号	f_X	f_Y	u	v
1	2 953.86	2 953.52	1 030.43	541.31
2	2 955.89	2 954.18	1 017.04	550.31
3	2 943.39	2 942.87	1 036.95	517.87

维度	平均绝对误差/mm	均方根误差/mm	最大误差/mm
X	0.110 0	0.142 0	0.320 9
Y	0.137 6	0.148 6	0.259 5
Z	0.141 6	0.166 6	0.392 2

方法	物距/m	真实值/mm	平均绝对误差/mm	均方根误差/mm	最大误差/mm
文献［5］	1.8	100	0.461 9	0.576 5	1.065 4
文献［15］	2.4	200	0.65		2.8
文献［16］	3	400		5.2	15
文中	1.8	502.63	0.174 5	0.135 8	0.449 5