Enhancing Vietnamese Medical Named Entity Recognition with BERT and CRF
Keywords:
named entitye recognition , vietnamese healthcare entity, deep learning, xlm-roberta, crfAbstract
Named Entity Recognition (NER) is used in the medical field, and this study looks into how it might be used to extract important information from unstructured textual data. For a variety of medical applications, it is critical to accurately identify things such as illnesses, available treatments, and healthcare initiatives. This paper presents a new method to improve NER performance in the medical domain by using Conditional Random Fields (CRF) in conjunction with Bidirectional Encoder Representations from Transformers (BERT). Superior performance and accuracy in identifying medical entities are achieved by the BERT-CRF model, which combines the contextual awareness of BERT with the sequence modeling capabilities of CRF. We test different BERT iterations in this work, including Base-BERT, RoBERTa, and XLM-RoBERTa. We used two distinct datasets for our tests. Two datasets are available: one in English with writings that have complicated and ambiguous semantics, and the other in Vietnamese with medical texts gathered from the Ministry of Health's Electronic Portal in Vietnam. The outcomes show that NER performed admirably on both datasets, especially in the medical field. The model using the XLM-RoBERTa version gives the best results for the Vietnamese medical dataset. This indicates how well NER for medical entity extraction may be improved in terms of accuracy and stability by integrating XLM-RoBERTa-CRF. The results of this study improve the suitability of natural language processing techniques for use in the medical field and have broad potential applications.
Downloads
References
Bartolini, I., Moscato, V., Postiglione, M., Sperlì, G., & Vignali, A. (2023). Data augmentation via context similarity: An application to biomedical Named Entity Recognition. Information Systems, 119, 102291.
Sabane, M., Ranade, A., Litake, O., Patil, P., Joshi, R., & Kadam, D. (2023, May). Enhancing Low Resource NER using Assisting Language and Transfer Learning. In 2023 2nd International Conference on Applied Artificial Intelligence and Computing (ICAAIC) (pp. 1666-1671). IEEE.
Jarrar, M., Abdul-Mageed, M., Khalilia, M., Talafha, B., Elmadany, A., Hamad, N., & Omar, A. (2023). WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. arXiv preprint arXiv:2310.16153.
Ariyanto, A. D. P., Fatichah, C., & Purwitasari, D. (2023, July). Semantic Role Labeling for Information Extraction on Indonesian Texts: A Literature Review. In 2023 International Seminar on Intelligent Technology and Its Applications (ISITIA) (pp. 119-124). IEEE.
Rattanatamrong, P., Boonpalit, Y., & Boonnavasin, M. (2024). Utilising crowdsourcing and text mining to enhance information extraction from social media: A case study in handling COVID-19 supply requests in Thailand. Journal of Information Science, 01655515231220164.
Al Mamun, M. A., Azad, M. A. K., & Pramanik, M. I. (2023). Crime-Finder: A System for Extraction and Visualization of Crime Data from Bengali Online Newspaper Articles. In Applied Informatics for Industry 4.0 (pp. 97-108). Chapman and Hall/CRC.
Younes, Y., & Scherp, A. (2023). Question Answering versus Named Entity Recognition for Extracting Unknown Datasets. IEEE Access.
Kruff, A., & Tran, A. H. (2023, July). Billie-newman at semeval-2023 task 5: Clickbait classification and question answering with pre-trained language models, named entity recognition and rule-based approaches. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023) (pp. 1542-1550).
Vági, R. (2023). How Could Semantic Processing and Other NLP Tools Improve Online Legal Databases?. TalTech Journal of European Studies, 13(2), 138-151.
Shafqat, S., Anwar, Z., Javaid, Q., & Ahmad, H. F. (2023). NER Sequence Embedding of Unified Medical Corpora to Incorporate Semantic Intelligence in Big Data Healthcare Diagnostics. Qeios.
Szczepanek, R. (2023). A deep learning model of spatial distance and named entity recognition (SD-NER) for flood mark text classification. Water, 15(6), 1197.
Kutbi, M. (2023). Named Entity Recognition Utilized to Enhance Text Classification while Preserving Privacy. IEEE Access.
Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25(1), 44-56.
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. (2018). Artificial intelligence in radiology. Nature Reviews Cancer, 18(8), 500-510.
Du‐Harpur, X., Watt, F. M., Luscombe, N. M., & Lynch, M. D. (2020). What is AI? Applications of artificial intelligence to dermatology. British Journal of Dermatology, 183(3), 423-430.
Navarro, D. F., Ijaz, K., Rezazadegan, D., Rahimi-Ardabili, H., Dras, M., Coiera, E., & Berkovsky, S. (2023). Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review. International Journal of Medical Informatics, 105122.
Li, M., & Zhang, R. (2023). How far is language model from 100% few-shot named entity recognition in medical domain. arXiv preprint arXiv:2307.00186.
Miah, M. S. U., Sulaiman, J., Sarwar, T. B., Islam, S. S., Rahman, M., & Haque, M. S. (2023, July). Medical named entity recognition (medner): A deep learning model for recognizing medical entities (drug, disease) from scientific texts. In IEEE EUROCON 2023-20th International Conference on Smart Technologies (pp. 158-162). IEEE.
Das, A., & Garain, U. (2014). Crf-based named entity recognition@ icon 2013. arXiv preprint arXiv:1409.8008.
Ekbal, A., & Bandyopadhyay, S. (2010). Named entity recognition using support vector machine: A language independent approach. International Journal of Electrical and Computer Engineering, 4(3), 589-604.
Higashiyama, S., Mathieu, B., Seki, K., & Uehara, K. (2015). Cost-sensitive structured perceptron incorporating category hierarchy for named entity recognition. Journal of Information and Communication Technology, 14, 1-20.
Jimmy, L., Nongmeikappam, K., & Naskar, S. K. (2023). Bilstm-crf Manipuri ner with character-level word representation. Arabian Journal for Science and Engineering, 48(2), 1715-1734.
Arslan, S. (2024). Application of BiLSTM-CRF model with different embeddings for product name extraction in unstructured Turkish text. Neural Computing and Applications, 1-12.
Guo, B., & Liu, H. (2023). Integration of natural and deep artificial cognitive models in medical images: BERT-based NER and relation extraction for electronic medical records. Frontiers in Neuroscience, 17, 1266771.
Song, Z., Xu, W., Liu, Z., Chen, L., & Su, H. (2023, August). A BERT-Based Named Entity Recognition Method of Warm Disease in Traditional Chinese Medicine. In 2023 IEEE 18th Conference on Industrial Electronics and Applications (ICIEA) (pp. 1226-1231). IEEE.
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., ... & Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116.
Ma, R., Wang, X., Zhou, X., Zhang, Q., & Huang, X. J. (2023, December). Towards Building More Robust NER datasets: An Empirical Study on NER Dataset Bias from a Dataset Difficulty View. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 4616-4630).
Li, Y., Zhou, K., Qiao, Q., Wang, Q., & Li, Q. (2024). Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approach. arXiv preprint arXiv:2402.14948.
Giachelle, F., Irrera, O., & Silvello, G. (2021). MedTAG: a portable and customizable annotation tool for biomedical documents. BMC Medical Informatics and Decision Making, 21(1), 352.
Mahajan, D., Liang, J. J., Tsou, C. H., & Uzuner, Ö. (2023). Overview of the 2022 n2c2 shared task on contextualized medication event extraction in clinical notes. Journal of Biomedical Informatics, 144, 104432.
Jingru, L., Yang, S., Rui, J., Yipeng, Z., Yong, L., & Jingdong, M. (2020). A BiLSTM-CRF Model for Protected Health Information in Chinese. Data Analysis and Knowledge Discovery, 4(10), 124-133.
Hamad, R. M., & Abushaala, A. M. (2023, May). Medical Named Entity Recognition in Arabic Text using SVM. In 2023 IEEE 3rd International Maghreb Meeting of the Conference on Sciences and Techniques of Automatic Control and Computer Engineering (MI-STA) (pp. 200-205). IEEE.
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4), 1234-1240.
Huang, K., Altosaar, J., & Ranganath, R. (2019). Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342.
Cevik, M., Mohammad Jafari, S., Myers, M., & Yildirim, S. (2023). Sequence Labeling for Disambiguating Medical Abbreviations. Journal of Healthcare Informatics Research, 7(4), 501-526.
Hanqing, Z. H. A. O., Li, Y., & Zhang, S. (2024). Symptom Extraction of Internal Medicine Diseases of Traditional Chinese Medicine Based on BERT-BiLSTM-CRF Model.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
