Customer Service Knowledge Recommendation Large Model Construction Driven by Intent Understanding

doi:10.12141/j.issn.1000-565X.240191

Abstract

Abstract:

With the deepening application of artificial intelligence technology in the field of customer service, telecommunications operators have raised higher standards for the accuracy of AI service knowledge recommendations. To enhance the efficiency and accuracy of knowledge recommendation in telecommunications operators’AI customer service systems, this paper proposed a large-scale customer service knowledge recommendation model driven by intent understanding. Firstly, the synonym and dialogue sequence keyword extraction model was employed to identify key terms in user queries. These keywords were then matched with questions in a standard question bank using semantic similarity comparison techniques to generate the most relevant standard questions. Additionally, a generative agent technology framework was utilized to construct and enrich the standard question bank, enabling the automatic generation of knowledge questions. The extracted standard questions were input into the ChatGLM2-6B large language model, which has been pre-trained and aligned with human preferences, further improving the accuracy of knowledge recommendations. The experimental results show that after the introduction of the standard question bank, the accuracy of the intelligent recommendation system in specific industry knowledge domains significantly increased from 74.8% to 85.9%. Multiple sets of comparative experimental results further validate the effectiveness of the strategy of establishing a standard question bank in improving accuracy. The large model discussed in this paper optimized the intelligent knowledge recommendation for operator AI customer service, providing new ideas and technical support for the knowledge recommendation in telecommunications operators’AI customer service systems. With this model, operators can more effectively understand and respond to customer inquiries, significantly enhancing the customer service experience.

Key words: intent understanding, AI customer service, large language model, keyword-based questioning, prompt example

CLC Number:

TP391.41

MA Xiaoliang, GAO Jie, LIU Ying, PEI Qingqi, ZHAO Ruqiang, YANG Bangxing, DENG Congjian. Customer Service Knowledge Recommendation Large Model Construction Driven by Intent Understanding[J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(3): 40-49.

Figures/Tables 10

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Table 2

Fig.8

References 18

1	黄勃，严非凡，张昊，等．推荐系统研究进展与应用［J］．武汉大学学报（理学版），2021，67（6）：503-516.
	HUANG Bo， YAN Feifan， ZHANG Hao，et al ．Pro-gress and Application of Recommendation System ［J］．Journal of Wuhan University（Natural Science Edition），2021，67（6）：503-516.
2	胡琪，朱定局，吴惠粦，等．智能推荐系统研究综述［J］．计算机系统应用，2022，31（4）：47-58.
	HU Qi， ZHU Ding-ju， WU Hui-lin，et al ．Survey on intelligent recommendation system［J］．Computer Systems Applications，2022，31（4）：47-58.
3	冷亚军，陆青，梁昌勇．协同过滤推荐技术综述［J］．模式识别与人工智能，2014，27（8）：720-734.
	LENG Ya-jun， LU Qing， LIANG Chang-yong ．Survey of recommendation based on collaborative filtering［J］．Pattern Recognition and Artificial Intelligence，2014，27（8）：720-734.
4	刘海涛，赵卫东．基于知识模式挖掘的流程知识推荐系统［J］．计算机集成制造系统，2017，23（2）：396-403.
	LIU Haitao， ZHAO Weidong ．Process-oriented know-ledge recommendation by mining knowledge patterns［J］．Computer Integrated Manufacturing Systems，2017，23（2）：396-403.
5	赵晔辉，柳林，王海龙，等．知识图谱推荐系统研究综述［J］．计算机科学与探索，2023，17（4）：771-791.
	ZHAO Yehui， LIU Lin， WANG Hailong，et al ．Survey of knowledge graph recommendation system research［J］．Journal of Frontiers of Computer Science and Technology，2023，17（4）：771-791.
6	秦琪琦，张月琴，王润泽，等．基于知识图谱的层次粒化推荐方法［J］．计算机科学，2022，49（8）：64-69.
	QIN Qiqi， ZHANG Yueqin， WANG Runze，et al ．Hie-rarchical granulation recommendation method based on knowledge graph［J］．Computer Science，2022，49（8）：64-69.
7	YUAN S， ZHAO H， DU Z，et al ．WuDaoCorpora：a super large-scale Chinese corpora for pre-training language models［J］．AI Open，2021，2：65-68.
8	MAATOUK A， PIOVESAN N， AYED F，et al ．Large language models for telecom：forthcoming impact on the industry［EB/OL］．（2023-08-11）［2024-04-14］．.
9	陈彬，张荣梅，张琦．DCFM：基于深度学习的混合推荐模型［J］．计算机工程与应用，2021，57（3）：150-155.
	CHEN Bin， ZHANG Rongmei， ZHANG Qi ．DCFM：hybrid recommendation model based on deep learning［J］．Computer Engineering and Applications，2021，57（3）：150-155.
10	马娟．智能客服在运营商中的主要应用场景探讨［J］．无线互联科技，2022，19（16）：124-127.
	MA Juan ．Discussion on main application scenarios of intelligent customer service in operators［J］．Wireless Internet Technology，2022，19（16）：124-127.
11	马晓亮，安玲玲，邓从健，等．基于行业词表的自动语音转写后优化技术［J］．华南理工大学学报（自然科学版），2023，51（8）：118-125.
	MA Xiaoliang， AN Lingling， DENG Congjian，et al ．Translation optimization technology of automatic speech recognition based on industry-specific vocabulary［J］．Journal of South China University of Technology（Natural Science Edition），2023，51（8）：118-125.
12	PARK J S， O’BRIEN J C， CAI C J，et al ．Generative agents：interactive simulacra of human behavior ［EB/OL］（2023-08-06）［2024-04-14］．.
13	WILLIAMS R， HOSSEINICHIMEH N， MAJUMDAR A，et al ．Epidemic modeling with generative agents ［EB/OL］．（2023-07-10）［2024-04-14］．.
14	杨兴锐，赵寿为，张如学，等．改进BERT词向量的BiLSTM-Attention文本分类模型［J］．传感器与微系统，2023，42（10）：160-164.
	YANG Xingrui， ZHAO Shouwei， ZHANG Ruxue，et al ．BiLSTM-attention text classification model of improved BERT word vector［J］．Transducer and Microsystem Technologies，2023，42（10）：160-164.
15	马晓亮，安玲玲，朱栩，等．基于小样本学习的关键词提取方法及装置：CN202211459952.0［P］．2023-12-29.
16	DU Z， QIAN Y， LIU X，et al ．GLM：general language model pretraining with autoregressive blank infilling ［EB/OL］．（2021-03-18）［2024-04-14］．.
17	LEE H， PHATALE S， MANSOOR H， et al ．RLAIF vs．RLHF：scaling reinforcement learning from human feedback with AI feedback ［EB/OL］．（2023-09-01）［2024-04-14］．.
18	SUN T， ZHANG X， HE Z，et al ．MOSS：an open conversational large language model［J］．Machine Inte-lligence Research，2024，21（5）：888-905.

模型	A/%
模型	通用行业	垂直行业
GPT-4	68.7	51.3
ChatGPT	54.4	32.6
MOSS	31.1	13.9
SQL模糊查询协同过滤文中模型	33.8 37.3 40.5	42.5 47.1 85.9

模型	P/%	R/%	F₁/%
BiLSTM-CRF	83.8	89.9	88.0
KeyBERT	75.8	67.3	70.6
RoBERTa	79.1	80.7	86.4
S & DS模型	86.2	98.5	91.9

[1]	XIAN Jin, XU Xiaoru, XIAN Yunting, XIAN Chuhua. Image Inpainting Algorithm Based on Hybrid Encoding and Mask Space Modulation [J]. Journal of South China University of Technology(Natural Science Edition), 2025, 53(3): 31-39.
[2]	LIU Ning, HUA Tianbiao, WANG Gao, et al. A Batch Scheduling Method of Flexible Job-Shop with Partially Out-of-Ordered Execute Operation [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 51-63.
[3]	MA Biyun, FAN Yihua, LIU Jiaojiao. Blood Flow Velocity Estimation Method Based on Multi-Frequency Pulse Sampling [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 22-30.
[4]	HU Guanghua, TU Qianxi. Surface Defect Detection Method for Industrial Products Based on Photometric Stereo and Dual Stream Feature Fusion Network [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(10): 112-123.
[5]	HU Guanghua, OU Meitong, LI Zhendong. Multi-Object Recognition and 6-DoF Pose Estimation Based on Synthetic Datasets [J]. Journal of South China University of Technology(Natural Science Edition), 2024, 52(4): 42-50.
[6]	LUO Yutao, GAO Qiang. Traffic Sign Detection Based on Channel Attention and Feature Enhancement [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(12): 64-72.
[7]	LI Haiyan, YIN Haolin, LI Peng, et al.. Image Inpainting Algorithm Based on Dense Feature Reasoning and Mix Loss Function [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 99-109.
[8]	LIU Yijun, WANG Jiada, ZHONG Shijie, et al.. Fast Multi-View Clustering Based on Uniform Label Matrix [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 110-119.
[9]	WANG Shiyong, QIAN Guokang, LI Di, et al.. Real-Time Template Matching Method for Edge Features [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(9): 1-10.
[10]	HAN Le, JIANG Yihua. Robust TruncatedL1-L2Total Variation Sparse Restoration Models [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 45-53.
[11]	LIN Zhijian, DING Yongqiang, YANG Xiuzhi, et al. Parallel Pipeline Hardware Design of Intra Rate-Distortion Optimization Prediction Mode in HEVC [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 95-103.
[12]	MA Biyun, WU Gang, LIU Jiaojiao, et al. Low Complexity Blood Flow Velocity Estimation Algorithm via Sparse Pulse Sampling [J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 63-69.
[13]	YI Qingming, LÜ Renyi, SHI Min, et al. Lightweight Object Detection Combined with Multi-Scale Dilated-Convolution and Multi-Scale Deconvolution [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(12): 41-48.
[14]	HE Kai, LIU Zhiguo, LI Dashuang, et al. Non-rigid Point Set Registration Using Neighborhood Structure and Driving Force Criterion [J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(4): 73-80.
[15]	ZHANG Yan, GAO Zijian, XU Changkang, et al. Classification Method of Tactile Pressure Footprint Based on Fusion Distribution Graph Network [J]. Journal of South China University of Technology (Natural Science Edition), 2022, 50(1): 91-100.