Journal of South China University of Technology (Natural Science Edition) ›› 2021, Vol. 49 ›› Issue (1): 47-57.doi: 10.12141/j.issn.1000-565X.200210

Special Issue: 2021年计算机科学与技术

• Computer Science & Technology • Previous Articles     Next Articles

Collaborative Score Prediction Method for Non-Random Missing Data

GU Wanrong1 XIE Xianfen2 ZHANG Ziye3 MAO Yijun1 LIANG Zaoqing1 HE Yichen1   

  1. 1. School of Mathematics and Information,South China Agricultural University,Guangzhou 510642,Guangdong,China; 2. School of Economics,Jinan University,Guangzhou 510632,Guangdong,China; 3. School of Mathematics,South China University of Technology,Guangzhou 510640,Guangdong,China
  • Received:2020-05-06 Revised:2020-06-17 Online:2021-01-25 Published:2021-01-01
  • Contact: 毛宜军 ( 1979-) ,男,博士,讲师,硕士生导师,主要从事大数据分析和生物信息学研究。 E-mail:yijunmao@163.com
  • About author:古万荣 ( 1982-) ,男,博士,讲师,硕士生导师,主要从事互联网大数据处理与分析和推荐模型研究。E-mail: guwanrong@scau.edu.cn
  • Supported by:
    Supported by the National Key R&D Program of China ( 2017YFC1601701) ,the Science and Technology Planning Project of Guangdong Province ( 2018A070712021) ,the National Key Project of Statistical Science Research ( 2019LZ37) and the Philosophy and Social Sciences Planning Project of Guangdong Province ( GD18CXW01,GD19CGL34)

Abstract: Most score prediction studies are based on the assumption that the missing values are random. However, the missing data of the score matrix of the actual on-line recommendation system is non-random. Incorrect assumptions about the missing data can lead to biased parameter estimation and prediction. In order to improve the accuracy of non-random missing score matrix filling,the internal principle of user and item score matrix was analyzed in this paper. It presents a method to transform the score matrix of user and object into the equivalent bilateral block diagonal matrix by row or column transformation. Then the matrix decomposition method was applied to different blocks to decompose and predict the score,making local data update and decomposition become a reality. The experimental results on the public test dataset show that the proposed method can improve the score filling effect,solve the problem of non-random score missing effectively,and improve the prediction accuracy of the recommendation system. The improved block matrix also has a better speedup ratio in the distributed processing experiment,which shows that the proposed method has better scalability.

Key words: matrix decomposition, recommendation system, singular value decomposition, score prediction

CLC Number: