The Grasshopper Optimization Technique for Hate Speech Detection on Multimodal Dataset
Keywords:
Hate speech, Multimodal, Optimization, Deep Learning, Machine LearningAbstract
Interest in multi-modal issues has increased recently, from image captioning to addressing visual questions and beyond. Online hate speech is a huge social problem nowadays, harming both individuals and society. One new kind of hostile communication, known as a "hateful meme," has arisen among them. Hate speech affects how minorities are viewed by society, even though it is not always connected to hate crimes Despite hate crimes being a public health problem, hate speech is not one in the United States. Identifying hate speech as a public health concern degrades the effects on victims and downplays hate crimes, while clearly recognizing hate speech as such downplays the act and calls for action, such as the creation of new rules or the allocation of resources to assist victims. So, hate speech identification with optimized way helps in international cooperation. Hateful memes were constructed with both text captions and images to reflect users' intentions; therefore, it is impossible to identify them with precision by only looking at the embedded text captions or photos. Identifying hate speech in multimodal memes is the new challenge set for multimodal categorization proposed in this work. Due to the addition of challenging cases to the dataset, it is difficult to rely on unimodal signals and only multimodal models may be successful. With the help of an effective feature selection technique, a grasshopper optimization algorithm GOA, and a transfer learning model VGG16, we concentrated on the identification of hate speech in multi-modal memes in this study. We attempt to resolve the Facebook Meme Challenge, a problem of binary classification that asks if a meme is hateful or not. We also include the feature selection optimization approach in addition to the multi-modal representations derived from the pre-trained model. Our model (GOA+VGG16) outperformed the other baseline models in a public test set by achieving an accuracy of 87 percent on the hateful meme identification task after using optimization algorithms and the VGG16 model and linking to a random forest (RF) classifier.
Downloads
References
Gunasekara and I. Nejadgholi, “A Review of Standard Text Classification Practices for Multi-label Toxicity Identification of Online Content,” 2nd Work. Abus. Lang. Online - Proc. Work. co-located with EMNLP 2018, pp. 21–25, 2018, doi: 10.18653/v1/w18-5103.
Yadav and Dinesh Kumar Vishwakarma, “MRT-Net: Auto-adaptive weighting of manipulation residuals and texture clues for face manipulation detection,” Expert Syst. with Appl., vol. 232, 2023, doi: https://doi.org/10.1016/j.eswa.2023.120898.
S. Gite et al., “Textual Feature Extraction Using Ant Colony Optimization for Hate Speech Classification,” Big Data Cogn. Comput., vol. 7, no. 1, 2023, doi: 10.3390/bdcc7010045.
F. Yang et al., “Exploring Deep Multimodal Fusion of Text and Photo for Hate Speech Classification,” no. 2017, pp. 11–18, 2019, doi: 10.18653/v1/w19-3502.
D. Kiela et al., “The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes,” arXiv:2005.04790v3 [cs.AI], pp. 1–17, 2021, [Online]. Available: http://arxiv.org/abs/2005.04790.
Chhabra and D. K. Vishwakarma, “A literature survey on multimodal and multilingual automatic hate speech identification,” Multimed. Syst., no. 0123456789, 2023, doi: 10.1007/s00530-023-01051-8.
M. Ali, F. A. Ghaleb, M. S. Mohammed, F. J. Alsolami, and A. I. Khan, “Web-Informed-Augmented Fake News Detection Model Using Stacked Layers of Convolutional Neural Network and Deep Autoencoder,” Mathematics, vol. 11, no. 9, 2023, doi: 10.3390/math11091992.
H. Aka Uymaz and S. Kumova Metin, “Vector based sentiment and emotion analysis from text: A survey,” Eng. Appl. Artif. Intell., vol. 113, no. May, p. 104922, 2022, doi: 10.1016/j.engappai.2022.104922.
S. Gandhi et al., “Scalable detection of offensive and non-compliant content / logo in product images,” Proc. - 2020 IEEE Winter Conf. Appl. Comput. Vision, WACV 2020, pp. 2236–2245, 2020, doi: 10.1109/WACV45572.2020.9093454.
M. Zampieri et al., “SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020),” Proc. Int. Work. Semant. Eval., no. OffensEval, 2020.
S. Poria, N. Majumder, D. Hazarika, E. Cambria, A. Gelbukh, and A. Hussain, “Multimodal Sentiment Analysis: Addressing Key Issues and Setting Up the Baselines,” IEEE Intell. Syst., vol. 33, no. 6, pp. 17–25, 2018, doi: 10.1109/MIS.2018.2882362.
S. Poria, I. Chaturvedi, E. Cambria, and A. Hussain, “Convolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis,” 2016 IEEE 16th Int. Conf. Data Min., pp. 439–448, 2017, doi: 10.1109/icdm.2016.0055.
S. Poria, E. Cambria, N. Howard, G. Bin Huang, and A. Hussain, “Fusing audio, visual and textual clues for sentiment analysis from multimodal content,” Neurocomputing, vol. 174, pp. 50–59, 2016, doi: 10.1016/j.neucom.2015.01.095.
E. T. Niu, S. Zhu, L. Pang, “Sentiment analysis on multi-view social data,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), p. 9517, 2016, doi: http://dx.doi.org/10.1007/978-3-319-27674-8_2.
F. Huang, X. Zhang, Z. Zhao, J. Xu, and Z. Li, “Image–text sentiment analysis via deep multimodal attentive fusion,” Knowledge-Based Syst., vol. 167, pp. 26–37, 2019, doi: 10.1016/j.knosys.2019.01.019.
H. Ma, J. Wang, L. Qian, and H. Lin, “HAN-ReGRU: hierarchical attention network with residual gated recurrent unit for emotion recognition in conversation,” Neural Comput. Appl., vol. 33, no. 7, pp. 2685–2703, 2021, doi: 10.1007/s00521-020-05063-7.
S. Poria, E. Cambria, and A. Gelbukh, “Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis,” Conf. Proc. - EMNLP 2015 Conf. Empir. Methods Nat. Lang. Process., no. September, pp. 2539–2544, 2015, doi: 10.18653/v1/d15-1303.
Chhabra and D. Kumar, “A Truncated SVD Framework for Online Hate Speech Detection on the ETHOS Dataset,” pp. 1–4, 2023.
Chhabra and D. K. Vishwakarma, “Fuzzy and Machine learning Classifiers for Hate Content Detection : A Comparative Analysis,” pp. 22–25, 2022.
W. A. Arentz and B. Olstad, “Classifying offensive sites based on image content,” Comput. Vis. Image Underst., vol. 94, no. 1–3, pp. 295–310, 2004, doi: 10.1016/j.cviu.2003.10.007.
P. Kakumanu, S. Makrogiannis, and N. Bourbakis, “A survey of skin-color modeling and detection methods,” Pattern Recognit., vol. 40, no. 3, pp. 1106–1122, 2007, doi: 10.1016/j.patcog.2006.06.010.
Tian, X. Zhang, W. Wei, and X. Gao, “Color pornographic image detection based on color-saliency preserved mixture deformable part model,” Multimed. Tools Appl., vol. 77, no. 6, pp. 6629–6645, 2018, doi: 10.1007/s11042-017-4576-2.
Chhabra and D. K. Vishwakarma, “Multimodal hate speech detection via multi-scale visual kernels and knowledge distillation architecture,” Eng. Appl. Artif. Intell., vol. 126, no. PB, p. 106991, 2023, doi: 10.1016/j.engappai.2023.106991.
Kiela et al., “The hateful memes challenge: Detecting hate speech in multimodal memes,” Adv. Neural Inf. Process. Syst., vol. 2020-Decem, no. NeurIPS, pp. 1–14, 2020.
R. Gomez et al., “Exploring Hate Speech Detection in Multimodal Publications,” https://help.twitter.com/en/ rules-and-policies/hateful-conduct-policy.2020
S Suryawanshi et al. "Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text” Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, pages 32–41 Language Resources and Evaluation Conference (LREC 2020), Marseille, 11–16 May 2020
American College of Physicians: American college of physicians says hate crimes are public health issue. Last Modified August 14. https://www.acponline.org/acp-newsroom/american-college-of-physicians-says-hate-crimes-are-public-health-issue (2017). Accessed 19 July 2022
Annu Dhankhar et al. A Survey on Multimodal hate speech Detection published in 2023 IEEE HTC conference
Annu Dhankhar et al. Feature extraction from text using grasshopper optimization algorithm for identifying hate speech published in 2023 ICAICCIT
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
