关键词: 话题模型, 子话题, 层次聚类, 隐狄利克雷分布, 话题检测

Abstract: The topics generated by the topic model have the problem of semantic difference when using the latent Dirichlet allocation (LDA) to detect topic． The LDA model will generate generic topics with broad meanings，and the parameter K is set by human experience． These problems will lead to a mixed topic situation with multiple sub- topics in the modeling results． To solve the problems above，a subtopic detection algorithm was carried out by using a kind of document feature word sequence based on the hierarchical clustering algorithm． The algorithm was applied to solve the problems that the LDA model classification result is too coarse and low value of public opinion monito- ring caused by the generalization of hot topic detection results． Firstly，the LDA model's results were optimized by filtering two kinds of matrixs，i. e． ，the topic-word distribution and the document-word distribution． Then the over- lapping topics were detected and merged，and the generic topics and mixed topics were detected by using the densi- ty between documents． Finally，the hierarchical clustering algorithm was used to find the subtopics under a topic． The experimental results show that the detection of subtopics in this method can reflect the characteristics of hot topics at a deeper level，which is convenient for public opinion monitoring analysis． Compared with Single-Pass algo- rithm and K-means algorithm，the results obtained by this method are more effective． The selection strategy of K is robust to subtopic detection algorithms based on hierarchical clustering．

Key words: topic model, subtopic, hierarchical clustering, latent Dirichlet allocation, topic detection