Detecting Depressive Tweets by Weighted Voting Ensemble Model of Attention Based Bi LSTM and BERT with Transfer Learning
Keywords:
Bi-LSTM, BERT, Depression, transfer learning, twitter, word embeddingAbstract
A prevalent mental illness in today's world is depression. Only a small portion of the millions of people who experience depression seek adequate medical care. Prolonged depression without effective medical treatment can lead to suicide, which makes early intervention a necessity. Nowadays, people share their inner feelings through social media platforms such as Twitter, Facebook, etc., so they can be used wisely for the early detection of depression among its users. Current approaches, despite integrating machine learning techniques, frequently struggle with challenges related to accuracy and effectively capturing exact patterns present in textual data. We suggest an ensemble of standard word embedding with Bi-LSTM and advanced language model BERT with transfer learning using weighted voting to seek improved performance by resolving the current limitations and scores an accuracy of 97.4%. The novelty lies in the comprehensive utilization of advanced techniques to process and analyze social media content, contributing to early detection efforts and augmenting mental health support initiatives.
Downloads
References
M. Kerr, “Depression (major depressive disorder),” Healthline, 23-Sep-2020. [Online]. Available: https://www.healthline.com/health/ clinical-depression . [Accessed: 30-Apr-2023].
L. A. Pratt, B. G. Druss, R. W. Manderscheid, and E. R. Walker, “Excess mortality due to depression and anxiety in the United States: results from a nationally representative survey,” Gen. Hosp. Psychiatry, vol. 39, pp. 39–45, 2016.
PricewaterhouseCoopers, “The Socio-economic impact of untreated mental illness,” PwC. [Online].Available:https://www.pwc.com/m1 /en/publications/socio-economic-impact-untreated-mental-illness.html.
“Depressive disorder (depression),”Who.int. [Online]. Available:https://www.who.int/news-room/fact-sheets/detail/ depression . [Accessed: 01-May-2023].
R. D. Goodwin, L. C. Dierker, M. Wu, S. Galea, C. W. Hoven, and A. H. Weinberger, “Trends in U.s. depression prevalence from 2015 to 2020: The widening treatment gap,” Am. J. Prev. Med., vol. 63, no. 5, pp. 726–733, 2022.
Clement, J.: Twitter: number of monthly active user 2010–2019(2019),https://www.statista.com /statistics/282087/number-of-monthly-active-twitter-users/
Benamara, Farah and Moriceau, Véronique and Mothe,Josiane and Ramiandrisoa, Faneva and He, Zhaolong “Automatic Detection of Depressive Users in Social Media.”(2018) In: Conférence francophone en Recherche d'Informationet Applications (CORIA), 16 May 2018 - 18 May 2018 (Rennes, France).
K. Sailunaz and R. Alhajj, “Emotion and sentiment analysis from Twitter text,” J. Comput. Sci., vol. 36, no. 101003, p. 101003, 2019.
K. Chakraborty, S. Bhatia, S. Bhattacharyya, J. Platos, R. Bag, and A. E. Hassanien, “Sentiment Analysis of COVID-19 tweets by Deep Learning Classifiers-A study to show how popularity is affecting accuracy in social media,” Appl. Soft Comput., vol. 97, no. 106754, p. 106754, 2020.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
Yong Chen, Bin Zhou, Weina Zhang, Wenjie Gong, and Guangfu Sun. 2018. Sentiment Analysis Based on Deep Learning and Its Application in Screening for Perinatal Depression. In DSC. IEEE, 451–456.
Khafaga, D. S., Auvdaiappan, M., Deepa, K., Abouhawwash, M., & Karim, F. K. (2023). Deep Learning for Depression Detection Using Twitter Data. INTELLIGENT AUTOMATION AND SOFT COMPUTING, 36(2), 1301-1313.
M. R. Khan, S. Z. Rizvi, A. Yasin, and M. Ali, “Depression Analysis of Social Media Activists Using the Gated Architecture Bi-LSTM,” 2021 Int. Conf. Cyber Warf. Secur. ICCWS 2021 - Proc., no. 2014, pp. 76–81, 2021, doi: 10.1109/ICCWS53234.2021.9703014.
H. Kour and M. K. Gupta, “An hybrid deep learning approach for depression prediction from user tweets using feature-rich CNN and bi-directional LSTM,” Multimed. Tools Appl., vol. 81, no. 17, pp. 23649–23685, 2022.
Guangyao Shen, Jia Jia, Liqiang Nie, Fuli Feng, Cunjun Zhang, Tianrui Hu, Tat-Seng Chua, and Wenwu Zhu. 2017. Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution.. In IJCAI. 3838–3844.
S. Munoz and C. A. Iglesias, “A text classification approach to detect psychological stress combining a lexicon-based feature framework with distributional representations,” Inf. Process. Manag., vol. 59, no. 5, p. 103011, 2022, doi: 10.1016/j.ipm.2022.103011.
P. KVTKN and T. Ramakrishnudu, “A novel method for detecting psychological stress at tweet level using neighborhood tweets,” J. King Saud Univ. - Comput. Inf. Sci., no. xxxx, 2021, doi: 10.1016/j.jksuci.2021.08.015.
J. S. L. Figuerêdo, A. L. L. M. Maia, and R. T. Calumby, “Early depression detection in social media based on deep learning and underlying emotions,” Online Soc. Networks Media, vol. 31, no. August 2021, p. 100225, 2022, doi: 10.1016/j.osnem.2022.100225.
S. G. Burdisso, M. Errecalde, and M. Montes-y-Gómez, “A text classification framework for simple and effective early depression detection over social media streams,” Expert Syst. Appl., vol. 133, pp. 182–197, 2019, doi: 10.1016/j.eswa.2019.05.023.
K. Elshakankery and M. F. Ahmed, “HILATSA: A hybrid Incremental learning approach for Arabic tweets sentiment analysis,” Egypt. Informatics J., vol. 20, no. 3, pp. 163–171, 2019, doi: 10.1016/j.eij.2019.03.002.
H. Zogan, I. Razzak, S. Jameel, and G. Xu, “DepressionNet: A novel summarization boosted deep framework for depression detection on social media,” arXiv [cs.LG], 2021.
M. Rizwan, M. F. Mushtaq, U. Akram, A. Mehmood, I. Ashraf, and B. Sahelices, “Depression classification from tweets using small deep transfer learning language models,” IEEE Access, vol. 10, pp. 129176–129189, 2022.
J. Pool-Cen, H. Carlos-Martínez, G. Hernández-Chan, and O. Sánchez-Siordia, “Detection of depression-related tweets in Mexico using crosslingual schemes and knowledge distillation,” Healthcare (Basel), vol. 11, no. 7, 2023.
P. Arora and P. Arora, “Mining twitter data for depression detection,” in 2019 International Conference on Signal Processing and Communication (ICSC), 2019.
S. J. Malla and A. P.J.A., “COVID-19 outbreak: “An ensemble pre-trained deep learning model for detecting informative tweets,” Appl. Soft Comput., vol. 107, p. 107495, 2021, doi: 10.1016/j.asoc.2021.107495.
Prakash, K. Agarwal, S. Shekhar, T. Mutreja, and P. S. Chakraborty, “An ensemble learning approach for the detection of depression and mental illness over twitter data,” Proc. 2021 8th Int. Conf. Comput. Sustain. Glob. Dev. INDIACom 2021, pp. 565–570, 2021, doi: 10.1109/INDIACom51348.2021.00100.
D. Sunitha, R. K. Patra, N. V. Babu, A. Suresh, and S. C. Gupta, “Twitter sentiment analysis using ensemble based deep learning model towards COVID-19 in India and European countries,” Pattern Recognit. Lett., vol. 158, pp. 164–170, 2022, doi: 10.1016/j.patrec.2022.04.027.
S. Symeonidis, D. Effrosynidis, and A. Arampatzis, “A comparative evaluation of pre-processing techniques and their interactions for twitter sentiment analysis,” Expert Syst. Appl., vol. 110, pp. 298–310, 2018.
T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” 2013.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional Transformers for language understanding,” 2018.
H. K. Cho, “Twitter Depression Dataset.” 09-Aug-2021.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
