Cognitive Diagnosis Model Integrating Forgetting and Importance of Knowledge Points

doi:10.12141/j.issn.1000-565X.220279

Abstract

Abstract:

Intelligence education is the key research direction of artificial intelligence. The most important is to describe the students’ cognitive process by ultilizing the knowledge points in the test questions. Aiming at the problem that the cognitive diagnosis model is insufficient for mining students, test questions and their interactive information, this study proposed a cognitive diagnosis model integrating forgetting and the importance of knowledge points. According to the historical interaction between the test questions and knowledge points, the model introduces forgetting factors in combination with the difficulty information of knowledge points, thus alleviates the problem of insufficient information mining for students. Through the attention mechanism, the importance information of the test questions to the knowledge points was obtained to alleviate the problem of insufficient information mining of the test questions. Learning the interaction relation between students and test questions through Transformer alleviates the problem of insufficient interaction information between students and test questions. The results of experiments carried out on the classic dataset show that the accuracy A_cc, root mean square error (RMSE), and the area under curve (AUC) values of this method on the Math1, Math2, and Assistment datasets are 0.716, 0.445, 0.776, 0.725, 0.432, 0.807, 0.741, 0.427, 0.779, respectively. Compared with other existing models, the proposed method has better results. The proposed method illustrates the importance of knowledge importance and timeliness for cognitive modeling.

Key words: cognitive diagnosis, attention mechanism, transformer, importance of knowledge points, forgetting information

CLC Number:

TP391

LIU Yupeng, ZHANG Lei. Cognitive Diagnosis Model Integrating Forgetting and Importance of Knowledge Points[J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(5): 54-62.

Figures/Tables 10

Fig.1

Fig.2

Table 1

Table 2

Fig.3

Table 3

Fig.4

Table 4

Table 5

Fig.5

References 18

1	NICHOLS P D， JOLDERSMA K ．Cognitive diagnostic assessment for education：theory and applications［J］．Journal of Educational Measurement，2008，45（4）：407- 411．
2	TORRE J ．DINA model and parameter estimation：a didactic［J］．Journal of Educational and Behavioral Statistics，2009，34（1）：115-130．
3	EMBRETSON S E， REISE S P ．Item response theory［M］．London：Psychology Press，2013： 56-87．
4	RECKAS M ．Multi-dimensional item response theory［J］．Handbook of Statistics，2009，26（6）： 607-642．
5	CHENG S， LIU Q， CHEN E，et al ．DIRT：deep learning enhanced item response theory for cognitive diagnosis ［C］∥ Proceedings of the 28th ACM International Conference on Information and Knowledge Management．Beijing：ACM，2019：2397-2400．
6	ZHU T， LIU Q， HUANG Z，et al ．MT-MCD：a multi-task cognitive diagnosis framework for student assessment ［C］∥ Proceedings of the 23rd International Conference on Database Systems for Advanced Applications．Gold Coast：Springer，2018：318-335．
7	LIU Q， WU R， CHEN E，et al ．Fuzzy cognitive diagnosis for modelling examinee performance［J］．ACM Transactions on Intelligent Systems and Technology，2018，9（4）：48/1-26．
8	XU J， LI Q， LIU J，et al ．Leveraging cognitive diagnosis to improve peer assessment in MOOCs［J］．IEEE Access，2021，9：50466-50484．
9	PIECH C， BASSEN J， HUANG J，et al ．Deep knowledge tracing［C］∥ Proceedings of the 28th International Conference on Neural Information Processing Systems．Cambridge：MIT Press，2015：505-513．
10	GIERL M J， CUI Y， HUNKA S ．Using connectionist models to evaluate examinees’ response patterns to achievement tests［J］．Journal of Modern Applied Statistical Methods，2008，7（1）：234-245．
11	WANG F， LIU Q， CHEN E，et al ．Neural cognitive diagnosis for intelligent education systems ［C］∥ Proceedings of the 34th AAAI Conference on Artificial Intelligence．Menlo Park：AAAI，2020：6153-6161．
12	李晓光，魏思齐，张昕，等．LFKT：学习与遗忘融合的深度知识追踪模型［J］．软件学报，2021，32（3）：818-830．
	LI Xiao-guang， WEI Si-qi， ZHANG Xin，et al ．LFKT：deep knowledge tracing model with learning and forgetting behavior merging［J］．Journal of Software，2021，32（3）：818-830．
13	HUANG Z， LIU Q， CHEN Y，et al ．Learning or forgetting？ A dynamic approach for tracking the knowledge proficiency of students［J］．ACM Transactions on Information Systems，2020，38（2）：19/1-33．
14	WU R， LIU Q， LIU Y，et al ．Cognitive modelling for predicting examinee performance ［C］∥ Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence．Buenos Aires：AAAI，2015：1006-1097．
15	FENG M， HEFFERNAN N， KOEDINGER K ．Addressing the assessment challenge with an online system that tutors as it assesses［J］．User Modeling and User-Adapted Interaction，2009， 19（3）：243-266．
16	XIONG X， ZHAO S， Van INWEGEN E G，et al ．Going deeper with deep knowledge tracing［C］∥Proceedings of the 9th International Conference on Educational Data Mining．Raleigh：International Educational Data Mining Society，2016：545-550．
17	SALAKHUTDINOV R， MNIH A ．Probabilistic matrix factorization［C］∥ Proceedings of the 20th International Conference on Neural Information Processing Systems．Red Hook：Curran Associates Inc．，2008：1257-1264．
18	CHENG Y， LI M， CHEN H，et al ．Neural cognitive modeling based on the importance of knowledge point for student performance prediction［C］∥ Proceedings of 16th International Conference on Computer Science & Education．Lancaster：IEEE，2021：495-499．

数据集	学生数	试题数	知识点数	作答记录	试题平均知识点数
Assistment	4 163	17 746	123	324 572	1.19
Math1	4 209	15	11	63 135	3.20
Math2	3 911	16	16	62 576	3.25

模型	A_cc			RMSE			AUC
模型	Math1	Math2	Assistment	Math1	Math2	Assistment	Math1	Math2	Assistment
DINA	0.593	0.592	0.650	0.487	0.475	0.467	0.686	0.683	0.676
IRT	0.612	0.610	0.674	0.480	0.471	0.464	0.702	0.699	0.685
PMF	0.605	0.603	0.659	0.483	0.472	0.471	0.701	0.698	0.732
MIRT	0.623	0.619	0.693	0.473	0.466	0.466	0.707	0.701	0.713
DIRT	0.636	0.639	0.705	0.465	0.465	0.453	0.717	0.720	0.722
NeuralCD	0.677	0.688	0.719	0.460	0.454	0.439	0.740	0.760	0.735
IKNCD	0.687	0.697	0.734	0.452	0.444	0.438	0.754	0.777	0.762
FK-CD	0.716	0.725	0.741	0.445	0.432	0.427	0.776	0.807	0.779

模型	A_cc			RMSE			AUC
模型	Math1	Math2	Assistment	Math1	Math2	Assistment	Math1	Math2	Assistment
F-CD	0.681	0.703	0.735	0.453	0.442	0.438	0.760	0.781	0.767
K-CD	0.698	0.707	0.737	0.449	0.440	0.437	0.770	0.787	0.767
FK-CD_T	0.699	0.710	0.738	0.449	0.439	0.435	0.772	0.789	0.769
FK-CD	0.716	0.725	0.741	0.445	0.432	0.427	0.776	0.807	0.779

数据集	DOA
数据集	NeuralCD	IKNCD	FK-CD
Math1	0.667	0.673	0.691
Math2	0.617	0.625	0.637
Assistment	0.795	0.805	0.818

知识点	单位费率	单位转换	乘法	绝对值	加法	学生a	学生b
试题1	1	1	0	0	0	错	对
试题2	0	1	1	0	0	对	对
试题3	0	0	0	1	1	对	错