Hybrid Deep Learning Model-Based Approach for Sentiment Classification

Usha  G. R.; J. V.  Gorabal

Authors

Usha G. R. Assistant Professor, Department of Information Science and Engineering, Sri Dharmasthala Manjunatheshwara Institute of Technology, Ujire-574240, Affiliated to Visvesvaraya Technological University, Belagavi, Karnataka, India https://orcid.org/0000-0002-3557-1060
J. V. Gorabal Professor, Department of Computer Science and Engineering ATME College of Engineering, Mysuru, Karnataka, India Affiliated to Visvesvaraya Technological University, Belagavi, Karnataka, India

Keywords:

CNN, LSTM, GRU, BiGRU, Glove, Word2vec

Abstract

The sentiment analysis task is more complex considering the lack of relevant information in brief texts. Deep neural networks, like as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have been widely employed to extract information from data sentiment in recent years, with surprisingly good results. Though CNN can efficiently retrieve comparatively high features employing convolution and max-pooling layers, it cannot understand relationships' sequences. Parallelly bidirectional RNN models can extract contextual information and fail to extract local features. In this paper, integrated CNN and RNN models for sentiment analysis are examined to have the advantages of CNN's coarse grain local feature extraction and long-distance dependencies of RNNs. Particularly bidirectional LSTM and GRU networks associated with the convolution and max-pooling layer are used for sentiment analysis in SST-2 and movie review datasets. Two pre-trained word embedding techniques glove and word2vec are used. Experimental findings show that max performance is achieved at 93.44% for SST-2 and 95.42% for the movie review dataset using CNN BiGRU word2vec and CNN BiGRU glove, respectively.

Downloads

Download data is not yet available.

References

Ji, P., Zhang, H. Y., & Wang, J. Q. (2018). A fuzzy decision support model with sentiment analysis for items comparison in e-commerce: The case study of http://PConline. com. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(10), 1993-2004.

Zeng, D., Dai, Y., Li, F., Wang, J., & Sangaiah, A. K. (2019). Aspect based sentiment analysis by a linguistically regularized CNN with gated mechanism. Journal of Intelligent & Fuzzy Systems, 36(5), 3971-3980.

Cambria, E., Das, D., Bandyopadhyay, S., & Feraco, A. (2017). Affective computing and sentiment analysis. In A practical guide to sentiment analysis (pp. 1-10). Springer, Cham.

García-Moya, L., Anaya-Sánchez, H., & Berlanga-Llavori, R. (2013). Retrieving product features and opinions from customer reviews. IEEE Intelligent Systems, 28(3), 19-27.

Reyes, A., & Rosso, P. (2012). Making objective decisions from subjective data: Detecting irony in customer reviews. Decision support systems, 53(4), 754-760.

Weichselbraun, A., Gindl, S., & Scharl, A. (2014). Enriching semantic knowledge bases for opinion mining in big data applications. Knowledge-based systems, 69, 78-85.

Pang, B., & Lee, L. (2004). A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. arXiv preprint cs/0409058.

Cho, H., Kim, S., Lee, J., & Lee, J. S. (2014). Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews. Knowledge-Based Systems, 71, 61-71.

Chen, L. S., Liu, C. H., & Chiu, H. J. (2011). A neural network based approach for sentiment classification in the blogosphere. Journal of Informetrics, 5(2), 313-322.

Dang, Y., Zhang, Y., & Chen, H. (2009). A lexicon-enhanced method for sentiment classification: An experiment on online product reviews. IEEE Intelligent Systems, 25(4), 46-53.

Moraes, R., Valiati, J. F., & Neto, W. P. G. (2013). Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Systems with Applications, 40(2), 621-633.

Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 649-657.

Zhang, L., Wang, S., & Liu, B. (2018). Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1253.

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.

Xu, G., Meng, Y., Qiu, X., Yu, Z., & Wu, X. (2019). Sentiment analysis of comment texts based on BiLSTM. Ieee Access, 7, 51522-51532.

Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural computation, 18(7), 1527-1554.

Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).

Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365.

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

Collobert, R., & Weston, J. (2008, July). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning (pp. 160-167).

Tripathy, A., Agrawal, A., & Rath, S. K. (2016). Classification of sentiment reviews using n-gram machine learning approach. Expert Systems with Applications, 57, 117-126.

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.

Huang, F., Li, X., Yuan, C., Zhang, S., Zhang, J., & Qiao, S. (2021). Attention-emotion-enhanced convolutional LSTM for sentiment analysis. IEEE Transactions on Neural Networks and Learning Systems.

Wu, O., Yang, T., Li, M., & Li, M. (2020). Two-Level LSTM for Sentiment Analysis With Lexicon Embedding and Polar Flipping. IEEE Transactions on Cybernetics.

Zhang, Y., & Wallace, B. (2015). A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820.

Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179-211.

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 1097-1105.

Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188.

Shen, Y., He, X., Gao, J., Deng, L., & Mesnil, G. (2014, April). Learning semantic representations using convolutional neural networks for web search. In Proceedings of the 23rd international conference on world wide web (pp. 373-374).

Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of machine learning research, 12(ARTICLE), 2493-2537.

Tam, S., Said, R. B., & Tanriöver, Ö. Ö. (2021). A ConvBiLSTM Deep Learning Model-Based Approach for Twitter Sentiment Classification. IEEE Access, 9, 41283-41293.

Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673-2681.

Zhou, J., Huang, J. X., Chen, Q., Hu, Q. V., Wang, T., & He, L. (2019). Deep learning for aspect-level sentiment classification: survey, vision, and challenges. IEEE access, 7, 78454-78483.

https://www.kaggle.com/atulanandjha/stanford-sentimenttreebank-v2-sst2.

https://www.cs.cornell.edu/people/pabo/movie-review-data/.