A combined Bi-LSTM-GPT Model for Arabic Sentiment Analysis
Keywords:
BiLSTM, AraGPT, Arabic sentiment analysis, deep learning, ensemble learningAbstract
This research investigates the efficacy of ensemble learning within the field of Arabic sentiment analysis. Ensemble learning, which combines predictions from multiple models to enhance accuracy, has shown promising results when compared to individual models. Hence, we propose an ensemble learning model that integrates two robust models: a Bidirectional Long Short-Term Memory (BiLSTM) model and a Generative pre-trained transformers (GPT) model. The GPT model has previously demonstrated effectiveness in various Arabic natural language processing (ANLP) tasks. To examine the performance of our ensemble model, we separately trained the BiLSTM and transformer-based model using three different datasets. We combined the models by aggregating their final probabilities for each class. Through multiple experiments, we compared the effectiveness of the proposed ensemble model with the standalone models. The results clearly indicate that the ensemble learning models outperform the standalone models in Arabic sentiment analysis. Specifically, the proposed ensemble model that demonstrated an accuracy increase of nearly 7% when compared to the best standalone model.
Downloads
References
Abdelali A, Darwish K, Durrani N, Mubarak H (2016) Farasa: A Fast and Furious Segmenter for Arabic. In: Proceedings of the 2016 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Demonstrations. Association for Computational Linguistics, San Diego, California, pp 11–16
Abu Farha I, Magdy W (2019) Mazajak: An Online Arabic Sentiment Analyser. In: Proceedings of the Fourth Arabic Natural Language Processing Workshop. Association for Computational Linguistics, Florence, Italy, pp 192–198
Abu Farha I, Magdy W (2021) A comparative study of effective approaches for Arabic sentiment analysis. Inf Process Manag 58:102438. https://doi.org/https://doi.org/10.1016/j.ipm.2020.102438
Al-Bayati AQ, Al-Araji AS, Ameen SH (2020) Arabic Sentiment Analysis (ASA) Using Deep Learning Approach. J Eng 26:85–93. https://doi.org/10.31026/j.eng.2020.06.07
Alshammari NF, Almansour AA (2020) Aspect-based Sentiment Analysis for Arabic Content in Social Media. In: 2nd International Conference on Electrical, Communication and Computer Engineering, ICECCE 2020. pp 1–6
Antoun W, Baly F, Hajj H (2020a) AraBERT: Transformer-based Model for Arabic Language Understanding. In: Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection. European Language Resource Association, Marseille, France, pp 9–15
Antoun W, Baly F, Hajj H (2020b) Araelectra: Pre-training text discriminators for arabic language understanding. arXiv Prepr arXiv201215516
Antoun W, Baly F, Hajj H (2020c) AraGPT2: Pre-Trained Transformer for Arabic Language Generation
Atoum JO, Nouman M (2019) Sentiment Analysis of Arabic Jordanian Dialect Tweets. Int J Adv Comput Sci Appl
Basiri ME, Nemati S, Abdar M, et al (2021) ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis. Futur Gener Comput Syst 115:279–294. https://doi.org/https://doi.org/10.1016/j.future.2020.08.005
Behera RK, Jena M, Rath SK, Misra S (2021) Co-LSTM: Convolutional LSTM model for sentiment analysis in social big data. Inf Process Manag 58:102435. https://doi.org/https://doi.org/10.1016/j.ipm.2020.102435
Blas SN (2018) Social Media and the Arab Spring
Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference. Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171–4186
El Karfi I, El Fkihi S, Faizi R (2018) A spectral clustering-based approach for sentiment classification in modern standard Arabic. In: MCCSIS 2018 - Multi Conference on Computer Science and Information Systems; Proceedings of the International Conferences on Big Data Analytics, Data Mining and Computational Intelligence 2018, Theory and Practice in Modern Computing 2018 and Connected Sma. pp 59–65
ElJundi O, Antoun W, El Droubi N, et al (2019) hULMonA: The Universal Language Model in Arabic. In: Proceedings of the Fourth Arabic Natural Language Processing Workshop. Association for Computational Linguistics, Florence, Italy, pp 68–77
Geetha MP, Karthika Renuka D (2021) Improving the performance of aspect based sentiment analysis using fine-tuned Bert Base Uncased model. Int J Intell Networks 2:64–69. https://doi.org/https://doi.org/10.1016/j.ijin.2021.06.005
Li W, Qi F, Tang M, Yu Z (2020) Bidirectional LSTM with self-attention mechanism and multi-channel features for sentiment classification. Neurocomputing 387:63–77. https://doi.org/https://doi.org/10.1016/j.neucom.2020.01.006
Li X, Roth D (2002) Learning question classifiers. In: COLING 2002: The 19th International Conference on Computational Linguistics. pp 1–7
Lunardi A, Viterbo J, Boscarioli C, et al (2016) Domain-tailored multiclass classification of user reviews based on binary splits. In: Meiselwitz G (ed) Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer International Publishing, Cham, pp 298–309
Moudjari L, Akli-Astouati K, Benamara F (2020) An algerian corpus and an annotation platform for opinion and emotion analysis. In: LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. European Language Resources Association, Marseille, France, pp 1202–1210
Mubarak AEH, Magdy W (2018) ArSAS: An Arabic Speech-Act and Sentiment Corpus of Tweets. In: Al-Khalifa H, University KS, Magdy KSAW, et al. (eds) Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Paris, France
Nabil M, Aly M, Atiya AF (2015) ASTD: Arabic sentiment tweets dataset. In: Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pp 2515–2519
Refaee E, Rieser V (2014) An Arabic twitter corpus for subjectivity and sentiment analysis. In: Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014. pp 2268–2273
Rosenthal S, Farra N, Nakov P (2018) SemEval-2017 Task 4: Sentiment Analysis in Twitter. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, pp 502–518
Soufan A (2019) Deep learning for sentiment analysis of Arabic text. In: ACM International Conference Proceeding Series
Wadhawan A (2021) Arabert and farasa segmentation based approach for sarcasm and sentiment detection in arabic tweets. arXiv Prepr arXiv210301679
Kamatchi, S. B. ., Agme, V. N. ., Premkumar, S., Prasad, K. ., V, D. G. ., & Gugan, I. . (2023). Enhancing Microcomputer Edge Computing for Autonomous IoT Motion Control. International Journal on Recent and Innovation Trends in Computing and Communication, 11(3), 58–67. https://doi.org/10.17762/ijritcc.v11i3.6202
Shanthi, D. N. ., & J, S. . (2022). Social Network Based Privacy Data Optimization Using Ensemble Deep Learning Architectures. Research Journal of Computer Systems and Engineering, 3(1), 62–66. Retrieved from https://technicaljournals.org/RJCSE/index.php/journal/article/view/43
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
