Journal of South China University of Technology (Natural Science Edition) ›› 2011, Vol. 39 ›› Issue (4): 1-6.doi: 10.3969/j.issn.1000-565X.2011.04.001

• Computer Science & Technology •     Next Articles

Static Index Pruning Based on Document Importance

Li Xiao-ming  Shan Dong-dong   

  1. School of Electronics Engineering and Computer Science,Peking University,Beijing 100871,China
  • Received:2011-01-09 Online:2011-04-25 Published:2011-03-01
  • Contact: 李晓明(1957-) ，男，教授，主要从事网络信息搜索与挖掘研究． E-mail:lxm@ pku.edu.cn
  • About author:李晓明(1957-) ，男，教授，主要从事网络信息搜索与挖掘研究．
  • Supported by:

    国家自然科学基金资助项目( 60933004) ; 广东省计算机网络重点实验室资助项目( CCNL200601) ; “核心电子器件、高端通用芯片及基础软件产品”国家科技重大专项项目( 2011ZX01042-001-001)

PDF

681

Abstract:

As the quality and importance of Web pages are both variable,paper proposes a static index pruning method which uses the web page importance to determine the ratio of information kept for each document. The result of experiments on GOV2 dataset show that ( 1) the proposed method greatly reduces the storage size and speeds up the search; ( 2) when the pruned index takes only 13% of the original size,P@ 10 and P@ 20 reach or exceed the baseline using full index; and ( 3) by using the proposed method,P@ 10,P@ 20  and MAP are all better than those of the traditional method at the same pruning level.

Key words: search engine, inverted index, static index pruning, document importance