Visual and Haptic Interaction Algorithm for Orthopedic Virtual Surgery

doi:10.12141/j.issn.1000-565X.250025

Abstract

Abstract:

Real-time visual and precise haptic interaction algorithms are critical for achieving accurate “tactile sensation” in virtual surgical training. In order to reduce storage space, improve computational efficiency, accurately calculate cutting forces during bone milling, and balance the visual and haptic interaction effect, this paper proposed a visual and haptic interaction algorithm based on the Tri-dexel model. Firstly, the Tri-dexel model was employed to represent the bone and the surgical milling tool. Real-time geometric deformation during the virtual bone milling was achieved through boolean operations and rapid surface reconstruction algorithms. Secondly, by integrating the geometric parameters of the surgical milling tool, a haptic interaction model based on the micro-element cutting force was proposed. This model utilizes the boolean operation results between the bone and surgical milling tool to quickly and accurately solve the instantaneous undeformed chip thickness. Thirdly, the cutting force coefficients were identified and the haptic interaction model was validated through milling experiments to achieve haptic rendering. Finally, an orthopedic virtual surgical training system was built based on the above-mentioned algorithms, and the interaction algorithm was tested and evaluated experimentally. The results show that the predicted forces align with experimental measurements, with an average force error of less than 7%. The visual and haptic interactive algorithm satisfies a visual refresh rate of 30 Hz and a haptic refresh rate of 1 kHz. The developed orthopedic virtual surgical training system provides users with a highly immersive virtual bone milling training experience that can effectively improve users’ hand-eye coordination.

Key words: virtual surgery, force feedback, haptic interaction, Tri-dexel model, cutting force

CLC Number:

TP391.9

WANG Qinghui, FANG Daoxin, CHI Zipeng, NI Jianlong, XIE Hailong, LI Jingrong, LI Chunhai. Visual and Haptic Interaction Algorithm for Orthopedic Virtual Surgery[J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(9): 76-85.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Table 1

Fig.11

Table 2

References 22

[1]	ZHANG X， SUN X， SUN W，et al ．Deformation expression of soft tissue based on BP neural network［J］．Intelligent Automation and Soft Computing，2022，32（2）：1041-1053.
[2]	TANG Y M， CHAU K Y， KWOK A P K，et al ．A systematic review of immersive technology applications for medical practice and education-trends，application areas，recipients，teaching contents，evaluation methods，and performance［J］．Educational Research Review，2022，35：100429/1-22.
[3]	ERNST M O， BANKS M S ．Humans integrate visual and haptic information in a statistically optimal fashion［J］．Nature，2002，415（6870）：429-433.
[4]	王成勇，陈志桦，陈华伟，等．生物骨材料切除理论研究综述［J］．机械工程学报，2021，57（11）：2-32.
	WANG Chengyong， CHEN Zhihua， CHEN Huawei，et al ．A review on cutting mechanism for bone material ［J］．Journal of Mechanical Engineering，2021，57（11）：2-32.
[5]	LIU J， ZHENG G， YAN W ．A framework of state estimation on laminar grinding based on the CT image-force model［J］．Sensors，2025，25（1）：238/1-28.
[6]	MATSUNAGA T， TAKANO S， SHIMONO T，et al ．Handheld haptic drill simulator using visual servoing system for axial force presentation［C］∥ Proceedings of 2023 IEEE the 32nd International Symposium on Industrial Electronics．Helsinki：IEEE，2023：1-6.
[7]	PENG X， ZHANG W ．A virtual sculpting system based on Triple Dexel models with haptics［J］．Computer-Aided Design and Applications，2009，6（5）：645-659.
[8]	SALISBURY K， CONTI F， BARBAGLI F ．Haptic rendering：introductory concepts［J］．IEEE Computer Graphics and Applications，2004，24（2）：24-32.
[9]	TAKABI B， TAI B L ．A review of cutting mechanics and modeling techniques for biological materials［J］．Medical Engineering & Physics，2017，45：1-14.
[10]	XING Q， CHEN J X， LI J，et al ．A real time haptic simulator of spine surgeries［C］∥ Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology．Beijing：ACM，2015：121-124.
[11]	CETIN A， BIRCAN D A ．3D pull-out finite element simulation of the pedicle screw-trabecular bone interface at strain rates［J］．Proceedings of the Institution of Mechanical Engineers Part H：Journal of Enginee-ring in Medicine，2022，236（1）：134-144.
[12]	AL-ABDULLAH K I A， ABDI H， LIM C P，et al ．Force and temperature modelling of bone milling using artificial neural networks［J］．Measurement，2018，116：25-37.
[13]	ARBABTAFTI M， MOGHADDAM M， NAHVI A，et al ．Physics-based haptic simulation of bone machining ［J］．IEEE Transactions on Haptics，2011，4（1）：39-50.
[14]	LIAO Z， AXINTE D， GAO D ．On modelling of cutting force and temperature in bone milling［J］．Journal of Materials Processing Technology，2019，266：627-638.
[15]	赵雪峰，杨勇，秦浩，等．刀具非对称刃口瞬时铣削力建模与实验研究［J］．华南理工大学学报（自然科学版），2021，49（12）：69-78，112.
	ZHAO Xuefeng， YANG Yong， QIN Hao，et al ．Experiment and modeling of the instantaneous milling force for asymmetric edge cutter［J］．Journal of South China University of Technology （Natural Science Edition），2021，49（12）：69-78，112.
[16]	TUKORA B， SZALAY T ．Multi-dexel based material removal simulation and cutting force prediction with the use of general-purpose graphics processing units［J］．Advances in Engineering Software，2012，43（1）：65-70.
[17]	XIA G， WANG J， YAO B，et al ．Tactile perception-based depth and angle control during robot-assisted bent bone grinding［J］．IEEE Transactions on Industrial Informatics，2024，20（1）：50-61.
[18]	WANG R， ZHANG S， ULLAH I，et al ．Quasistatic deflection analysis of slender ball-end milling cutter［J］．International Journal of Mechanical Sciences，2024，264：108807/1-21.
[19]	KAY L， KAJIYA T ．Ray tracing complex scenes［J］．ACM SIGGRAPH Computer Graphics，1986，20（4）：269-278.
[20]	WANG L， SI H， GUAN L，et al ．Comparison of different polynomial functions for predicting cutting coefficients in the milling process［J］．International Journal of Advanced Manufacturing Technology，2018，94（5/6/7/8）：2961-2972.
[21]	JIANG Z， QI X， SUN Y，et al ．Cutting depth monitoring based on milling force for robot-assisted laminectomy［J］．IEEE Transactions on Automation Science and Engineering，2020，17（1）：2-14.
[22]	LI J R， FU J L， WU S C，et al ．An active and passive combined gravity compensation approach for a hybrid force feedback device［J］．Proceedings of the Institution of Mechanical Engineers Part C：Journal of Mechanical Engineering Science，2021，235（19）：4368-4381.

计算方法	布尔运算/ms	力觉渲染/ms	总时间/ms
GPU	0.58	0.36	0.97
CPU	22.15	6.49	29.46

操作次数	完成时间/s	超出安全范围次数	偏离程度/mm	材料去除程度
1	73.13	14.7	9.77	3.94
2	60.88	13.9	9.46	4.08
3	68.94	10.8	9.31	3.87
4	55.29	11.2	8.78	3.65
5	54.33	9.4	8.59	3.12
6	39.12	8.8	7.09	2.96
7	32.38	6.4	7.15	2.37
8	35.76	5.6	7.10	2.53