骨科虚拟手术的视觉与力觉融合交互算法

doi:10.12141/j.issn.1000-565X.250025

华南理工大学学报(自然科学版) ›› 2025, Vol. 53 ›› Issue (9): 76-85.doi: 10.12141/j.issn.1000-565X.250025

骨科虚拟手术的视觉与力觉融合交互算法

王清辉¹^,², 方道鑫¹, 池梓鹏¹, 倪建龙¹, 谢海龙³, 李静蓉¹, 李春海⁴

^1.华南理工大学机械与汽车工程学院，广东广州 510640
^2.人工智能与数字经济广东省实验室（广州），广东广州 510335
^3.华南理工大学设计学院，广东广州 510006
^4.中山大学孙逸仙纪念医院骨外科，广东广州 510120

收稿日期:2025-01-20 出版日期:2025-09-25 发布日期:2025-03-21
通信作者: 李静蓉（1973—），女，博士，教授，主要从事虚拟仿真与VR交互研究。 E-mail:lijr@scut.edu.cn
作者简介:王清辉（1972—），男，博士，教授，主要从事CAD/CAM技术、VR交互研究。E-mail： wqh@scut.edu.cn
基金资助:
国家重点研发计划项目(2022YFB4500600);广东省自然科学基金项目(2022A1515010100);广东省自然科学基金项目(2024A1515010140)

Visual and Haptic Interaction Algorithm for Orthopedic Virtual Surgery

WANG Qinghui¹^,², FANG Daoxin¹, CHI Zipeng¹, NI Jianlong¹, XIE Hailong³, LI Jingrong¹, LI Chunhai⁴

^1.School of Mechanical and Automotive Engineering，South China University of Technology，Guangzhou 510640，Guangdong，China
^2.Guangdong Laboratory of Artificial Intelligence and Digital Economy （Guangzhou），Guangzhou 510335，Guangdong，China
^3.School of Design，South China University of Technology，Guangzhou 510006，Guangdong，China
^4.Orthopedic Surgery，Sun Yat-sen Memorial Hospital，Sun Yat-sen University，Guangzhou 510120，Guangdong，China

Received:2025-01-20 Online:2025-09-25 Published:2025-03-21
Contact: 李静蓉（1973—），女，博士，教授，主要从事虚拟仿真与VR交互研究。 E-mail:lijr@scut.edu.cn
About author:王清辉（1972—），男，博士，教授，主要从事CAD/CAM技术、VR交互研究。E-mail： wqh@scut.edu.cn
Supported by:
the National Key R & D Program of China(2022YFB4500600);the Natural Science Foundation of Guangdong Province(2022A1515010100)

摘要/Abstract

摘要：

实时的视觉和精细的力觉融合交互算法是实现虚拟手术训练中精准“手感”的关键。为减少存储空间，提高计算效率，精确计算骨铣削过程中的切削力，平衡视觉和力觉融合的交互效果，该文提出了一种基于Tri-dexel模型的视觉与力觉融合交互算法。首先，采用Tri-dexel模型表示骨与医用铣刀，通过布尔运算、快速表面重建及渲染算法实现虚拟骨铣削操作过程中的实时几何变形计算；接着，结合医用铣刀的几何参数，提出基于微元切削力的骨铣削力觉交互模型，利用骨与医用铣刀Tri-Dexel模型之间布尔运算的结果，快速精确求解瞬时未变形切屑厚度；然后，通过槽切实验完成对切削力系数的辨识，并对该力觉模型进行验证分析，实现虚拟骨铣削操作过程中的力觉渲染；最后，基于上述算法搭建骨科虚拟手术训练系统，通过实验对视觉与力觉融合交互算法进行测试与评估。结果表明：力觉模型的预测结果与实验测量结果吻合较好，力的平均相对误差在7%以下；该算法能够同时满足30 Hz的视觉刷新频率以及1 kHz的力觉刷新频率要求；所搭建的骨科虚拟手术训练系统能够为用户提供高沉浸感的虚拟骨铣削操作训练，可有效提高用户的手眼协调能力。

关键词: 虚拟手术, 力反馈, 力觉交互, Tri-dexel模型, 切削力

Abstract:

Real-time visual and precise haptic interaction algorithms are critical for achieving accurate “tactile sensation” in virtual surgical training. In order to reduce storage space, improve computational efficiency, accurately calculate cutting forces during bone milling, and balance the visual and haptic interaction effect, this paper proposed a visual and haptic interaction algorithm based on the Tri-dexel model. Firstly, the Tri-dexel model was employed to represent the bone and the surgical milling tool. Real-time geometric deformation during the virtual bone milling was achieved through boolean operations and rapid surface reconstruction algorithms. Secondly, by integrating the geometric parameters of the surgical milling tool, a haptic interaction model based on the micro-element cutting force was proposed. This model utilizes the boolean operation results between the bone and surgical milling tool to quickly and accurately solve the instantaneous undeformed chip thickness. Thirdly, the cutting force coefficients were identified and the haptic interaction model was validated through milling experiments to achieve haptic rendering. Finally, an orthopedic virtual surgical training system was built based on the above-mentioned algorithms, and the interaction algorithm was tested and evaluated experimentally. The results show that the predicted forces align with experimental measurements, with an average force error of less than 7%. The visual and haptic interactive algorithm satisfies a visual refresh rate of 30 Hz and a haptic refresh rate of 1 kHz. The developed orthopedic virtual surgical training system provides users with a highly immersive virtual bone milling training experience that can effectively improve users’ hand-eye coordination.

Key words: virtual surgery, force feedback, haptic interaction, Tri-dexel model, cutting force

中图分类号:

TP391.9

王清辉, 方道鑫, 池梓鹏, 倪建龙, 谢海龙, 李静蓉, 李春海. 骨科虚拟手术的视觉与力觉融合交互算法[J]. 华南理工大学学报(自然科学版), 2025, 53(9): 76-85.

WANG Qinghui, FANG Daoxin, CHI Zipeng, NI Jianlong, XIE Hailong, LI Jingrong, LI Chunhai. Visual and Haptic Interaction Algorithm for Orthopedic Virtual Surgery[J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(9): 76-85.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

图11

表2

参考文献 22

[1]	ZHANG X， SUN X， SUN W，et al ．Deformation expression of soft tissue based on BP neural network［J］．Intelligent Automation and Soft Computing，2022，32（2）：1041-1053.
[2]	TANG Y M， CHAU K Y， KWOK A P K，et al ．A systematic review of immersive technology applications for medical practice and education-trends，application areas，recipients，teaching contents，evaluation methods，and performance［J］．Educational Research Review，2022，35：100429/1-22.
[3]	ERNST M O， BANKS M S ．Humans integrate visual and haptic information in a statistically optimal fashion［J］．Nature，2002，415（6870）：429-433.
[4]	王成勇，陈志桦，陈华伟，等．生物骨材料切除理论研究综述［J］．机械工程学报，2021，57（11）：2-32.
	WANG Chengyong， CHEN Zhihua， CHEN Huawei，et al ．A review on cutting mechanism for bone material ［J］．Journal of Mechanical Engineering，2021，57（11）：2-32.
[5]	LIU J， ZHENG G， YAN W ．A framework of state estimation on laminar grinding based on the CT image-force model［J］．Sensors，2025，25（1）：238/1-28.
[6]	MATSUNAGA T， TAKANO S， SHIMONO T，et al ．Handheld haptic drill simulator using visual servoing system for axial force presentation［C］∥ Proceedings of 2023 IEEE the 32nd International Symposium on Industrial Electronics．Helsinki：IEEE，2023：1-6.
[7]	PENG X， ZHANG W ．A virtual sculpting system based on Triple Dexel models with haptics［J］．Computer-Aided Design and Applications，2009，6（5）：645-659.
[8]	SALISBURY K， CONTI F， BARBAGLI F ．Haptic rendering：introductory concepts［J］．IEEE Computer Graphics and Applications，2004，24（2）：24-32.
[9]	TAKABI B， TAI B L ．A review of cutting mechanics and modeling techniques for biological materials［J］．Medical Engineering & Physics，2017，45：1-14.
[10]	XING Q， CHEN J X， LI J，et al ．A real time haptic simulator of spine surgeries［C］∥ Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology．Beijing：ACM，2015：121-124.
[11]	CETIN A， BIRCAN D A ．3D pull-out finite element simulation of the pedicle screw-trabecular bone interface at strain rates［J］．Proceedings of the Institution of Mechanical Engineers Part H：Journal of Enginee-ring in Medicine，2022，236（1）：134-144.
[12]	AL-ABDULLAH K I A， ABDI H， LIM C P，et al ．Force and temperature modelling of bone milling using artificial neural networks［J］．Measurement，2018，116：25-37.
[13]	ARBABTAFTI M， MOGHADDAM M， NAHVI A，et al ．Physics-based haptic simulation of bone machining ［J］．IEEE Transactions on Haptics，2011，4（1）：39-50.
[14]	LIAO Z， AXINTE D， GAO D ．On modelling of cutting force and temperature in bone milling［J］．Journal of Materials Processing Technology，2019，266：627-638.
[15]	赵雪峰，杨勇，秦浩，等．刀具非对称刃口瞬时铣削力建模与实验研究［J］．华南理工大学学报（自然科学版），2021，49（12）：69-78，112.
	ZHAO Xuefeng， YANG Yong， QIN Hao，et al ．Experiment and modeling of the instantaneous milling force for asymmetric edge cutter［J］．Journal of South China University of Technology （Natural Science Edition），2021，49（12）：69-78，112.
[16]	TUKORA B， SZALAY T ．Multi-dexel based material removal simulation and cutting force prediction with the use of general-purpose graphics processing units［J］．Advances in Engineering Software，2012，43（1）：65-70.
[17]	XIA G， WANG J， YAO B，et al ．Tactile perception-based depth and angle control during robot-assisted bent bone grinding［J］．IEEE Transactions on Industrial Informatics，2024，20（1）：50-61.
[18]	WANG R， ZHANG S， ULLAH I，et al ．Quasistatic deflection analysis of slender ball-end milling cutter［J］．International Journal of Mechanical Sciences，2024，264：108807/1-21.
[19]	KAY L， KAJIYA T ．Ray tracing complex scenes［J］．ACM SIGGRAPH Computer Graphics，1986，20（4）：269-278.
[20]	WANG L， SI H， GUAN L，et al ．Comparison of different polynomial functions for predicting cutting coefficients in the milling process［J］．International Journal of Advanced Manufacturing Technology，2018，94（5/6/7/8）：2961-2972.
[21]	JIANG Z， QI X， SUN Y，et al ．Cutting depth monitoring based on milling force for robot-assisted laminectomy［J］．IEEE Transactions on Automation Science and Engineering，2020，17（1）：2-14.
[22]	LI J R， FU J L， WU S C，et al ．An active and passive combined gravity compensation approach for a hybrid force feedback device［J］．Proceedings of the Institution of Mechanical Engineers Part C：Journal of Mechanical Engineering Science，2021，235（19）：4368-4381.

计算方法	布尔运算/ms	力觉渲染/ms	总时间/ms
GPU	0.58	0.36	0.97
CPU	22.15	6.49	29.46

操作次数	完成时间/s	超出安全范围次数	偏离程度/mm	材料去除程度
1	73.13	14.7	9.77	3.94
2	60.88	13.9	9.46	4.08
3	68.94	10.8	9.31	3.87
4	55.29	11.2	8.78	3.65
5	54.33	9.4	8.59	3.12
6	39.12	8.8	7.09	2.96
7	32.38	6.4	7.15	2.37
8	35.76	5.6	7.10	2.53

骨科虚拟手术的视觉与力觉融合交互算法

Visual and Haptic Interaction Algorithm for Orthopedic Virtual Surgery

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 22

相关文章 4

编辑推荐

Metrics

本文评价

[1]	周俊杰, 张凌钰, 于洋, 等. 遥操作软体机械臂的末端力反馈系统分析[J]. 华南理工大学学报(自然科学版), 2022, 50(10): 132-139.
[2]	孔金星邓飞夏志辉李亮. 纯铁切屑形成特征及影响因素研究[J]. 华南理工大学学报（自然科学版）, 2016, 44(12): 7-13.
[3]	王佳庆王少华. 变形蔡氏电路中的混沌控制[J]. 华南理工大学学报(自然科学版), 2003, 31(5): 75-78.
[4]	邓文君, 夏伟, 张卫文, 等. 倒角刀刃切削过程中切削力的有限元法预测[J]. 华南理工大学学报(自然科学版), 2003, 31(10): 20-23.