Improve Offensive Language Detection with Ensemble Classifiers
DOI:
https://doi.org/10.18201/ijisae.2020261592Keywords:
BabelNet, ensemble classifiers, offensive language, short text classification, Twitter, Word2VecAbstract
Sharing content easily on social media has become an important communication choice in the world we live. However, in addition to the conveniences it provides, some problems have been emerged because content sharing is not bounded by predefined rules. Consequently, offensive language has become as a big problem for both social media and its users. In this article, detecting offensive language in short text messages on Twitter is aimed. As short texts do not include enough statistical information have drawbacks. To cope with these drawbacks of the short texts, semantic word expansion based on concepts and word-embeddings vectors are proposed. Then, for classification task, decision tree and decision tree based ensemble classifiers such as Adaptive Boosting, Bootstrap Aggregating, Random Forest, Extremely Randomized Decision Tree and Extreme Gradient Boosting algorithms are used. Also, the imbalanced dataset problem is solved by oversampling. Experiments on the dataset show that extremely randomized trees with word-embedding vectors as input achieved 85.66% F-score.
Downloads
References
K. Denecke and W. Nejdl, “How valuable is medical social media data? Content analysis of the medical web,” Inform Sciences, vol. 179, no. 12, pp. 1870–1880, May. 2009.
M. Wiegand, M. Siegel, and J. Ruppenhofer, “Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language,” in Proc. KONVENS, Vienna, Austria, 2018,
pp. 1–10.
M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar, “Predicting the Type and Target of Offensive Posts in Social Media,” in Proc. NAACL-HLT, Minneapolis, Minnesota, USA, 2019,
pp. 1415–1420.
M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar, “SemEval-2019 Task6: Identifying and Categorizing Offensive Language in Social Media (OffensEval),” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019,
pp. 75–86.
J. Mitrovic, B. Birkeneder, and M. Garnitzer, “nlpUP at SemEval-2019 Task6: A Deep Neural Language Model for Offensive Language Detection,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 722–726.
B. Pang, L. Lee, and S. Vaithyanathan, “Thumbs up? Sentiment classification using machine learning techniques,” in Proc. EMNLP, Philadelphia, USA, 2002, pp. 79–86.
P-W. Liang and B-R. Dai, “Opinion Mining on Social Media Data,” in Proc. MDM, Italy, 2013, pp. 91–96.
S. Sevim, S. İlhan Omurca, and E. Ekinci, “An Ensemble Model using a BabelNet Enriched Document Space for Twitter Sentiment Classification,” IJISAE, vol. 10, no. 1, pp. 24–31, Oct. 2018.
Z. Faguo, Z. Fan, Y. Bingru, and Y. Xingang, “Research on Short Text Classification Algorithm Based on Statistics and Rules,” in Proc. ISECS, NW Washington, DC, USA, 2010,
pp. 3–7.
I. Alsmadi and K. H. Gan, “Review of short-text classification,” IJWIS, vol. 15, no. 2, pp. 155–182, June. 2019.
J. Tang, X. Wang, H. Gao, X. Hu, and H. Liu, “Enriching short text representation in microblog for clustering,” Front Comput Sci, vol. 6, no. 1, pp. 88–101, Jan. 2012.
Z. Liu, W. Yu, W. Chen, S. Wang, and F. Wu, “Short Text Feature Selection for Micro-blog Mining,” in Proc. ACL-HLT, Portland, Oregon, USA, 2011, pp. 151–160.
L. Jiang, M. Yu, M. Zhou, X. Liu, and T. Zhao, “Target-dependent Twitter Sentiment Classification,” in Proc. CiSE, Wuhan, China, 2010, pp. 1–4.
K. Kim, B-S. Chung, Y. Choi, S. Lee, and J-Y. Juang, “Language independent semantic kernels for short-text classification,” Expert Syst Appl, vol. 41, no. 2, pp. 735–743, Feb. 2014.
P. Wang, J. Xu, B. Xu, C-H. Liu, H. Zhang, F. Wang, and H. Hao, “Semantic Clustering and Convolutional Neural Network for Short Text Categorization,” in Proc. ACL- IJCNLP, Beijing, China, 2015, pp. 352–357.
B. Wang, J. Xu, B. Xu, G. Tian, C-L. Liu and H. Hao, “Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification,” Neurocomputing, vol. 174, no. B, pp. 806–814, Feb. 2016.
N. Sotthisopha and P. Vateekul, “Improving Short Text Classification Using Fast Semantic Expansion on Multichannel Convolutional Neural Network,” in Proc. IEEE/ACIS SNPD, Busan Gwang'yeogsi · South Korea, 2018, pp. 182–187.
B. V. Dasarathy and B. V. Sheela, “A Composite Classifier System Design: Concepts and Methodology,” Proceedings of the IEEE, vol. 67, no. 5, pp. 708–713, May. 1979.
E. Ekinci and H. Takçı, “Comparing Ensemble Classifiers: Forensic Analysis of Electronic Mails,” in Proc. INSODE, Antalya, Turkey, 2013, pp. 167–173.
R. Xia, C. Zong, and S. Li, “Ensemble of feature sets and classification algorithms for sentiment classification,” Inform Sciences, vol. 181, no. 6, pp. 1138–1152, Mar. 2011.
M. Tutek, I. Sekulic, P. Gombar, I. Paljak, P. Culinovic, F. Boltuzic, M. Karan, D. Alagic, and J. Snajder, “TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble,” in Proc. SemEval, San Diego, California, 2016, pp. 464–468.
Z. H. Kilimci and S. İlhan Omurca, “A Comparison of Extended Space Forests for Classifier Ensembles on Short Turkish Texts,” in Proc. AC-EITAI, Prague, Czech Republic, 2017, pp. 96–104.
Z. H. Kilimci and S. İlhan Omurca, “Extended Feature Spaces Based Classifier Ensembles for Sentiment Analysis of Short Texts,” Inf Technol Control, vol. 47, no. 3, pp. 457–470, Sep. 2018.
P. Liu, W. Li, and L. Zou, “NULI at SemEval-2019 Task6: Transfer Learning for Offensive Language Detection using Bidirectional Transformer,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 87–91.
E. Kebriaei, S. Karimi, N. Sabri, and A. Shakery, “Emad at SemEval-2019 Task 6: Offensive Language Identification using Traditional Machine Learning and Deep Learning approaches,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 600–603.
P. Aggarwal, T. Horsmann, M. Wojatzki, and T. Zesch, “LTL-UDE at SemEval-2019 Task6: BERT and Two-Vote Classification for Categorizing Offensiveness,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 678–682.
P. Rani and A. Kr. Ojha, “KMI−Coling a tSemEval-2019 Task6: Exploring N-grams for Offensive Language detection,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 668–671.
G-V. Patraş, D-F. Lungu, D. Gifu, and D. Trandabat, “Hope at SemEval-2019 Task 6: Mining social media language to discover offensive language,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 635–638.
P. Kapil, A. Ekbal, and D. Das, “NLP at SemEval-2019 Task6: Detecting Offensive language using Neural Networks,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 587–592.
H. Bansal, D. Nagel, and A. Soloveva, “HAD-Tübingen at SemEval-2019 Task6: Deep Learning Analysis of Offensive Language on Twitter: Identification and Categorization,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 622–627.
L. Balasubramanian, H. S. Kumar, G. Bandlamudi, D. Sivasankaran, R. Sivanaiah, A. D. Suseelan, S. M. Rajendram, and M. T. N. Thanagathai, “TECHSSN at SemEval-2019 Task6: Identifying and Categorizing Offensive Language in Tweets using Deep Neural Networks,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 753–758.
D. Thenmozhi, B. S. Kumar, C. Aravindan and S. Srinethe, “SSN_NLP at SemEval-2019 Task6: Offensive Language Identification in Social Media using Traditional and Deep Machine Learning Approaches,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 739–744.
G. L. De la P. Sarracen and P. Rosso, “Deep Analyzer at SemEval-2019 Task6: A deep learning-based ensemble method for identifying offensive tweets,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 582–586.
J. Han, S. Wu, and X. Liu, “jhan014 at SemEval-2019 Task6: Identifying and Categorizing Offensive Language in Social Media,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 652–656.
B. Wang, X. Zhou, and X. Zhang, “YNUWB at SemEval-2019 Task6: K-max pooling CNN with average meta-embedding for identifying offensive language,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 818–822.
V. Indurthi, B. Syed, M. Shrivastava, M. Gupta, and V. Varma, “Fermi at SemEval-2019 Task6: Identifying and Categorizing Offensive Language in Social Media using Sentence Embeddings,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 611–616.
A. Oberstrass, J. Romberg, A. Stoll, and S. Conrad, “HHU at SemEval-2019 Task6: Context Does Matter-Tackling Offensive Language Identification and Categorization with ELMo,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 628–634.
S. D. Swamy, A. Jamatia, B. Gamback, and A. Das, “NIT_Agartala_NLP_Team at SemEval-2019 Task6: An Ensemble Approach to Identifying and Categorizing Offensive Language in Twitter Social Media Corpora,” in Proc. SemEval, Minneapolis, Minnesota, USA, 2019, pp. 696–703.
C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky, “The Stanford Corenlp Natural Language Processing Toolkit,” in Proc. ACL, Baltimore, Maryland, 2014, pp. 55–60.
E. Ekinci and S. İlhan Omurca, “Concept-LDA: Incorporating Babelfy into LDA for aspect extraction”, J. Inf. Sci., Accessed on: Apr., 29, 2019, DOI: 10.1177/0165551519845854, [Online].
D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin, “Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification,” in Proc. ACL, Baltimore, Maryland, 2014, pp. 1555–1565.
H. K. Kim, H. Kim, and S. Cho, “Bag-of-concepts: Comprehending document representation through clustering words in distributed representation,” Neurocomputing, vol. 266, pp. 336–352, Nov. 2017.
A. Ali, S. M. Shamsuddin, and A. L. Ralescu, “Classification with class imbalance problem: A Review,” IJASCA, vol. 7, no. 3, pp. 176–204, Nov. 2015.
M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera, “A Review on Ensembles for the Class Imbalance Problem: Bagging, Boosting-, and Hybrid-Based Approaches,” IEEE Trans. Syst. Man Cybern. B. Cybern., vol. 42, no. 4, pp. 463–484, Aug. 2011.
A. D. McDonald, J. D. Lee, C. Schwarz, and T. L. Brown, “Steering in a Random Forest: Ensemble Learning for Detecting Drowsiness-Related Lane Departures,” Hum. Factors, vol. 56, no. 5, pp. 986–998, Aug. 2014.
L. Rokach, “Ensemble-based classifiers,” Artif. Intell. Rev., vol. 33, pp. 1–39, Feb. 2014.
Y. Freund and R. E. Schapire, “Experiments with a New Boosting Algorithm,” in Proc. ICML, Bari, Italy, 1996, pp. 325–332.
E. Ekinci, S. İlhan Omurca, and N. Acun, “A Comparative Study on Machine Learning Techniques using Titanic Dataset,” in Proc. ICAT, Antalya, Turkey, 2018, pp. 411–416.
L. Breiman, “Bagging Predictors,” Mach.Learn., vol. 24, pp. 123–140, Sep. 1994.
L. Breiman, “Random Forests,” Mach.Learn., vol. 45, pp. 5–32, Sep. 2001.
S. İlhan Omurca, E. Ekinci, B. Çakmak, and S. G. Özkan, “Using Machine Learning Approaches for Prediction of the Types of Asthmatic Allergy across the Turkey,” DataSCI., vol. 2, no. 2, pp. 8–12, Dec. 2019.
P. Geurts, D. Ernst, and L. Wehenkel, “Extremely randomized trees,” Mach.Learn., vol. 63, pp. 3–42, Sep. 2006.
T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proc. KDD, San Francisco, CA, USA, 2016, pp. 785–794.
Z. Chen, F. Jiang, Y. Cheng, X. Gu, W. Liu, and J. Peng, “XGBoost Classifier for DDoS Attack Detection and Analysis in SDN-based Cloud,” in Proc. BigComp, Shanghai, China, 2018, pp. 251–256.
Downloads
Published
How to Cite
Issue
Section
License
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
