Adaptive Multiscale Transformer Network with Bi-LSTM-based Neural Machine Translation Model using Attention Vector for Named Entity Recognition with Adolescent Suicidal Text
Keywords:
Named Entity Recognition; Machine Translation; Adaptive Multiscale Transformer Network with Bidirectional Long Short Term Memory; Neural Machine Translation; Fitness Improved COOT; Trans-Bi-LSTM based Recognition;Abstract
Suicide thoughts impact language usage stated on the internet. Many at-risk people utilize social discussion sites offering information about similar tasks. Our study aims to share ongoing research on automatically identifying suicidal comments. Deep learning classification techniques identify adolescents with suicidal thoughts in their early stages. The text data is gathered from standard data sources. The obtained textual data undergoes data pre-processing to remove redundant and inappropriate data. The pre-processed text data is given as input to the Adaptive Multiscale Transformer Network with Bidirectional Long Short Term Memory (AMTN-Bi-LSTM) for Named Entity Recognition (NER). The developed AMTN-Bi-LSTM consists of a Transformer Unit and a Bi-LSTM unit. At first, the pre-processed text data is given to the Transformer Network for Neural Machine Translation (NMT). During the translation time, the identified named entities are to be monitored via the specific process of translation model that helps to improve the quality. This Transformer Network with an inbuilt self-attention mechanism produces the pre-processed text's attention vectors as output. This attention vector of the text is now given to the encoder section of the Bi-LSTM. The encoded vector is then fed to the decoder section of the Bi-LSTM, from which the essential suicidal text words are recognized. For improved recognition, the parameters in the developed AMTN-Bi-LSTM model are tuned with the help of the Fitness Improved COOT (FICOOT) algorithm. The recognized text is given as input to the encoder unit of the Trans-Bi-LSTM, from which the given text is classified as a non-suicidal or suicidal class. The potential operation of the developed NER model for suicidal word recognition is verified by comparing the recommended method with the conventional models regarding various performance metrics.
Downloads
References
Q. Qiu, Z. Xie, L. Wu, L.Tao and W. Li , "BiLSTM-CRF for geological named entity recognition from the geoscience literature," Earth Science Informatics, vol. 12, pp. 565–579, 2019.
S. K.Gorla, S. S. Tangeda, L. B. M. Neti and A. Malapati, "Telugu named entity recognition using bert," International Journal of Data Science and Analytics, vol. 14, pp. 127–140, 2022.
M. Affi, and C. Latiri, "BE-BLC: BERT-ELMO-Based Deep Neural Network Architecture for English Named Entity Recognition Task," Procedia Computer Science, vol. 192, pp. 168-181, 2021.
C. Wu, G. Luo, C. Guo, Y. Ren, A. Zheng, and C. Yang, "An attention-based multi-task model for named entity recognition and intent analysis of Chinese online medical questions," Journal of Biomedical Informatics, vol. 108, pp. 103511, 2020.
D. Peng, D. Zhang, C. Liu, and J. Lu, "BG-SAC: Entity relationship classification model based on Self-Attention supported Capsule Networks," Applied Soft Computing, vol. 91, pp. 106186, 2020.
Z. Li, J. Yang, X. Gou, and X. Qi, "Recurrent neural networks with segment attention and entity description for relation extraction from clinical texts," Artificial Intelligence in Medicine, vol. 97, pp. 9-18, 2019.
X. Meng and J. Zhang, "Anxiety Recognition of College Students Using a Takagi-Sugeno-Kang Fuzzy System Modeling Method and Deep Features," IEEE Access, vol. 8, pp. 159897-159905, 2020.
A. Kumar J, T. E. Trueman, and A. K. Abinesh, "Suicidal risk identification in social media," Procedia Computer Science, vol. 189, pp. 368-373, 2021.
T. Zhang, A. M. Schoene, and S. Ananiadou, "Automatic identification of suicide notes with a transformer-based deep learning model," Internet Interventions, vol. 25, pp. 100422, 2021.
G. Berkelmans, L. Schweren, S. Bhulai, R. V. D. Mei, and R. Gilissen, "Identifying populations at ultra-high risk of suicide using a novel machine learning method," Comprehensive Psychiatry, vol. 123, pp. 152380, 2023.
S.Ghosal, and A. Jain, "Depression and Suicide Risk Detection on Social Media using fastText Embedding and XGBoost Classifier," Procedia Computer Science, vol. 218, pp. 1631-1639, 2023.
N. J.C. Stapelberg, M. Randall, J. Sveticic, P. Fugelli, H. Dave, and K. Turner, "Data mining of hospital suicidal and self-harm presentation records using a tailored evolutionary algorithm," Machine Learning with Applications, vol. 3, pp. 100012, 15 2021.
D. Lekkas, R. J. Klein, and N. C. Jacobson, "Predicting acute suicidal ideation on Instagram using ensemble machine learning models," Internet Interventions, vol. 25, 100424, 2021.
J. Du, Y. Zhang, J. Luo, Y. Jia, Q. Wei, C. Tao and H. Xu, "Extracting psychiatric stressors for suicide from social media using deep learning," BMC Medical Informatics and Decision Making, vol. 18, no. 43, 2018.
S. Ghosh, A. Ekbal and P. Bhattacharyya, "Deep cascaded multitask framework for detection of temporal orientation, sentiment and emotion from suicide notes," Scientific Reports, vol. 12, no. 4457, 2022.
P. Burnap, G. Colombo, R. Amery, A. Hodorog, and J. Scourfield, "Multi-class machine classification of suicide-related communication on Twitter," Online Social Networks and Media, vol. 2, pp. 32-44, 2017.
A. Chadha, and B. Kaushik, "A Hybrid Deep Learning Model Using Grid Search and Cross-Validation for Effective Classification and Prediction of Suicidal Ideation from Social Network Data," New Generation Computing, vol. 40, pp. 889–914, 2022.
T. Ghosh, Md. H. A. Banna, Md. J. A. Nahian, M. N. Uddin, M. S. Kaiser, and M. Mahmud, "An attention-based hybrid architecture with explainability for depressive social media text detection in Bangla," Expert Systems with Applications, vol. 213, Part C, pp. 119007, 1 2023.
S. R. Laskar, B. Paul, P. Pakray, and S. Bandyopadhyay, "English-Assamese Multimodal Neural Machine Translation using Transliteration-based Phrase Augmentation Approach," Procedia Computer Science, vol. 218, pp. 979-988, 2023.
Y. Zhao, M. Komachi, T.Kajiwara, and C. Chu, "Region-attentive multimodal neural machine translation, Neurocomputing, vol. 476, pp. 1-13, 1 2022.
N. B. Allen, B. W. Nelson, D. Brent, and R. P. Auerbach, "Short-term prediction of suicidal thoughts and behaviors in adolescents: Can recent developments in technology and computational science provide a breakthrough?," Journal of Affective Disorders, vol. 250, pp. 163-169, 2019.
M. N. A. Ali and G. Tan, "Bidirectional Encoder–Decoder Model for Arabic Named Entity Recognition," Arabian Journal for Science and Engineering, vol. 44, pp. 9693–9701, 2019.
B. Priyamvada, S. Singhal, A. Nayyar, R. Jain, P. Goel, M. Rani and M. Srivastava, "Stacked CNN - LSTM approach for prediction of suicidal ideation on social media," Multimedia Tools and Applications, 2023.
A. Belouali, S. Gupta, V. Sourirajan, J. Yu, N. Allen, A. Alaoui, M. A. Dutton and Matthew J. Reinhard, "Acoustic and language analysis of speech for suicidal ideation among US veterans," BioData Mining, vol. 14, no. 11, 2021.
D. Kodati and R. Tene, "Identifying suicidal emotions on social media through transformer-based deep learning," Applied Intelligence, 2022.
A. S. Farag, M. Mohandes and A. A. Shaikh, "Diagnosing failed distribution transformers using neural networks," in IEEE Transactions on Power Delivery, vol. 16, no. 4, pp. 631-636, 2001.
R. Zhong, R. Wang, Y. Zou, Z. Hong and M. Hu, "Graph Attention Networks Adjusted Bi-LSTM for Video Summarization," IEEE Signal Processing Letters, vol. 28, pp. 663-667, 2021.
C. Li, J. Zheng, H. Pan, J. Tong and Y. Zhang, "Refined Composite Multivariate Multiscale Dispersion Entropy and Its Application to Fault Diagnosis of Rolling Bearing," IEEE Access, vol. 7, pp. 47663-47673, 2019.
N.Vassilina & S.Agnes & D. Marc, "Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation", Conference: Proceedings of the Second Workshop on Applying Machine Learning Techniques to Optimise the Division of Labour in Hybrid MT, pp. 1-16. 2012.
I. Naruei, and F. Keynia, "A new optimization method based on COOT bird natural life model," Expert Systems with Applications, vol. 183, pp. 115352, 30.
Z. Elgamal, A. Q. M. Sabri, M. Tubishat, D. Tbaishat, S. N. Makhadmeh and O. A. Alomari, "Improved Reptile Search Optimization Algorithm Using Chaotic Map and Simulated Annealing for Feature Selection in Medical Field," IEEE Access, vol. 10, pp. 51428-51446, 2022.
M. Obayya, J. M. Alsamri, M. A. Al-Hagery, A. Mohammed and M. A. Hamza, "Automated Cardiovascular Disease Diagnosis Using Honey Badger Optimization With Modified Deep Learning Model," IEEE Access, vol. 11, pp. 64272-64281, 2023, doi: 10.1109/ACCESS.2023.3286661.
B. M. Nguyen, T. Tran, T. Nguyen and G. Nguyen, "Hybridization of Galactic Swarm and Evolution Whale Optimization for Global Search Problem," IEEE Access, vol. 8, pp. 74991-75010, 2020.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
