BiLSTM-BiDAF Named Entity Recognition Based on Machine Reading Comprehension

doi:10.12141/j.issn.1000-565X.220013

Abstract

Abstract:

Named entity recognition is a fundamental task of natural language processing (NLP) and plays an important role in many downstream NLP tasks, including information extraction and machine translation, etc. The existing named entity recognition methods are usually based on sequence labeling and extract entities within a sentence independently. These methods ignore the semantic information between sentences. Named entity recognition methods based on machine reading comprehension encode important prior information about the entity class. It is easier to distinguish similar classification labels, which reduces the difficulty of model learning, but it still only models at the sentence level, ignoring the semantic information between sentences, which is easy to cause the problem of inconsistent entity labeling in different sentences. To this end, this paper extended the sentence-level named entity recognition to the text-level named entity recognition, and then proposed a BiLSTM-BiDAF named entity recognition model based on machine reading comprehension. First, to utilize the context information within the whole text, NEZHA pre-training language model was used to obtain information of the full text and local features were further captured through BiLSTM, so as to strengthen the model’s ability to capture locally dependent information. Then, a bidirectional attention flow was introduce to learn the semantic association between the text and entity category. Finally, to predict the position of entities in the text, a boundary detector based on the gating mechanism was design to strengthen the correlation of the entity boundary. At the same time, an answer count detector was establish to identify the unanswerable questions. Experimental results on the CCKS2020 Chinese electronic medical records dataset and CMeEE dataset show that our model can effectively identify document-level and sentence-level named entities, and F₁ can reach 84.76% and 57.35%, respectively.

Key words: bidirectional attention flow, bidirectional long short-term memory, named entity recognition, machine reading comprehension, natural language processing

CLC Number:

TP18

WANG Jie, XIA Xiaoming . BiLSTM-BiDAF Named Entity Recognition Based on Machine Reading Comprehension[J]. Journal of South China University of Technology(Natural Science Edition), 2022, 50(12): 80-88.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Table 2

Fig.5

Table 3

Table 4

Table 5

Table 6

References 21

1	SHEN Y， MA X， TAN Z，et al ．Locate and label：a two-stage identifier for nested named entity recognition ［C］∥ Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing．Stroudsburg：Association for Computational Linguistics，2021：2782-2794.
2	LI J， SUN A， HAN J，et al ．A survey on deep learning for named entity recognition ［J］．IEEE Transactions on Knowledge and Data Engineering，2022，34（1）：50-70.
3	刘奕洋，余正涛，高盛祥，等．基于机器阅读理解的中文命名实体识别方法［J］．模式识别与人工智能，2020，33（7）：653-659.
	LIU Yiyang， YU Zhengtao， GAO Shengxiang，et al ．Chinese named entity recognition method based on machine reading comprehension ［J］．Pattern Recognition and Artificial Intelligence，2020，33（7）：653-659．
4	HUANG Z， XU W， YU K ．Bidirectional LSTM-CRF models for sequence tagging ［EB/OL］．（2015-08-09）［2022-01-07］．.
5	HABIBI M， WEBER L， NEVES M，et al ．Deep lear-ning with word embeddings improves biomedical named entity recognition ［J］．Bioinformatics，2017，33（14）：i37-i48.
6	WANG X， ZHAMG Y， REN X，et al ．Cross-type biomedical named entity recognition with deep multi-task lear-ning ［J］．Bioinformatics，2019，35（10）：1745-1752.
7	YU H，KO Y ．Expansion of word representation for named entity recognition based on bidirectional LSTM CRFs ［J］．Journal of KIISE，2017，44（3）：306-313.
8	郭知鑫，邓小龙．基于BERT-BiLSTM-CRF的法律案件实体智能识别方法［J］．北京邮电大学学报，2021，44（4）：129-134.
	GUO Zhi-xin， DENG Xiao-long ．Intelligent identification method of legal case entity based on BERT-BiLSTM-CRF ［J］．Journal of Beijing University of Posts and Telecommunications，2021，44（4）：129-134.
9	REI M， CRICHTON G， PYYSALO S ．Attending to characters in neural sequence labeling models ［C］∥ Proceedings of the 26th International Conference on Computational Linguistics．Stroudsburg：Association for Computational Linguistics，2016：309-318.
10	DEVLIN J， CHANG M W， LEE K，et al ．BERT：pre-training of deep bidirectional transformers for language understanding ［C］∥ Proceedings of the 2019 Confe-rence of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies．Minneapolis：Association for Computational Linguistics，2019：4171-4186.
11	RADMARD P， FATHULLAH Y， LIPANI A ． Subsequence based deep active learning for named entity recognition ［C］∥ Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing．Stroudsburg：Association for Computational Linguistics，2021：4310-4321.
12	MA R， PENG M， ZHANG Q，et al ．Simplify the usage of lexicon in Chinese NER ［C］∥ Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics．Stroudsburg：Association for Computational Linguistics，2020：5951-5960.
13	LU T， GUI Y， GAO Z ．Document-level named entity recognition with Q-network ［C］∥ Proceedings of the 16th Pacific Rim International Conference on Artificial Intelligence．Cham：Springer，2019：164-178.
14	LEVY O， SEO M， CHOI E，et al ．Zero-shot relation extraction via reading comprehension ［C］∥ Proceedings of the 21st Conference on Computational Natural Language Learning．Stroudsburg：Association for Computational Linguistics，2017：333-342.
15	LI X， YIN F， SUN Z，et al ．Entity-relation extraction as multi-turn question answering ［C］∥ Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics．Minneapolis：Association for Computational Linguistics，2019：1340-1350.
16	LI X， FENG J， MENG Y，et al ．A unified MRC framework for named entity recognition ［C］∥ Procee-dings of the 58th Annual Meeting of the Association for Computational Linguistics．Stroudsburg：Association for Computational Linguistics，2020：5849-5859.
17	XUE M， YU B， ZHANG Z，et al ．Coarse-to-fine pre-training for named entity recognition ［C］∥ Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing．Stroudsburg：Association for Computational Linguistics，2020：6345-6354.
18	HU A， DOU Z， NIE J Y，et al ．Leveraging multi-token entities in document-level named entity recognition ［C］∥ Proceedings of the 34th AAAI Conference on Artificial Intelligence．Menlo Park：AAAI，2020：7961-7968.
19	WEI J， REN X， LI X，et al ．NEZHA：neural contextualized representation for Chinese language understanding ［EB/OL］．（2019-08-31）［2022-01-07］．.
20	ZHANG W， JIANG S， ZHAO S，et al ．A BERT-BiLSTM-CRF model for Chinese electronic medical records named entity recognition ［C］∥ Proceedings of 2019 the 12th International Conference on Intelligent Computation Technology and Automation．Xiangtan：IEEE，2019：166-169.
21	STRUBELL E， VERGA P， BELANGER D，et al ．Fast and accurate entity recognition with iterated dilated convolutions ［C］∥ Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing．Copenhagen：Association for Computational Linguistics，2017：2670-2680.

数据集	原始文本条数	最长文本长度	文本平均长度	实体平均长度	实体个数
训练集	1 487	1 664	403	6	25 800
测试集	100	1 194	416	6	1 653

数据集	有答案数	无答案数	总数
训练集	5 780	3 142	8 922
测试集	412	188	600

实体类别	构建的问题
疾病和诊断	找出疾病和诊断，例如癌症、病变、炎症、增生、肿瘤
实验室检验	找出实验室检验，例如血型、比率、酶、蛋白、pH
手术	找出手术，例如切除、检查、根治、电切、穿刺、移植
药物	找出药物，例如胶囊
解剖部位	找出解剖部位，例如胃、肠、肝、肺、胸、臀
影像检查	找出影像检查，例如CT、CR、MRI、彩超、X光、造影

模型	CCKS2020			CMeEE
模型	p/%	r/%	F₁/%	p/%	r/%	F₁/%
NEZHA-CRF	79.07	81.93	80.47	56.14	54.95	55.54
NEZHA-IDCNN-CRF	81.88	80.05	80.96	55.09	56.02	55.55
BERT-BiLSTM-CRF	82.80	81.67	82.22	55.48	52.11	53.74
NEZHA-BiLSTM-CRF	83.75	83.48	83.61	56.39	54.88	55.63
NEZHA-MRC-baseline	79.36	81.16	80.23	55.99	51.71	53.76
MRC-NER	85.98	83.57	84.76	58.26	56.47	57.35

实体类别	p/%	r/%	F₁/%
疾病和诊断	84.00	79.16	81.51
影像检查	82.47	94.12	87.91
药物	93.55	90.22	91.86
解剖部位	86.96	84.57	85.74
手术	86.79	85.98	86.38
实验室检验	77.71	74.73	76.19