Sentiment Analysis Based Direction Prediction in Bitcoin using Deep Learning Algorithms and Word Embedding Models

Keywords: Bitcoin, deep learning, FastText, long short-term memory networks, sentiment analysis

Abstract

Sentiment analysis is a considerable research field to analyze huge amount of information and specify user opinions on many things and is summarized as the extraction of users’ opinions from the text. Like sentiment analysis, Bitcoin which is a digital cryptocurrency also attracts the researchers considerably in the fields of economics, cryptography, and computer science. The purpose of this study is to forecast the direction of Bitcoin price by analysing user opinions in social media such as Twitter. To our knowledge, this is the very first attempt which estimates the direction of Bitcoin price fluctuations by using deep learning and word embedding models in the state-of-the-art studies. For the purpose of estimating the direction of Bitcoin, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long-short term memory networks (LSTMs) are used as deep learning architectures and Word2Vec, GloVe, and FastText are employed as word embedding models in the experiments. In order to demonstrate the contibution of our work, experiments are carried out on English Twitter dataset. Experiment results show that the usage of FastText model as a word embedding model outperforms other models with 89.13% accuracy value to estimate the direction of Bitcoin price.

Downloads

Download data is not yet available.

References

V. M. Prieto, S. Matos, M. Alvarez, F. Cacheda, and J. L. Oliveira, “Twitter: a good place to detect health conditions”, PloS one, vol. 9, no. 1., pp. 1-11, Jan. 2014.

HILEMAN, Garrick; RAUCHS, Michel. Global cryptocurrency benchmarking study. Cambridge Centre for Alternative Finance, 2017, 33.

Madan, Isaac, Shaurya Saluja, and Aojia Zhao. "Automated bitcoin trading via machine learning algorithms." URL: http://cs229. stanford. edu/proj2014/Isaac% 20Madan 20 (2015).

SHAH, Devavrat; ZHANG, Kang. Bayesian regression and Bitcoin. In: 2014 52nd annual Allerton conference on communication, control, and computing (Allerton). IEEE, 2014. p. 409-414.

JANG, Huisu; LEE, Jaewook. An empirical study on modeling and prediction of bitcoin prices with bayesian neural networks based on blockchain information. Ieee Access, 2017, 6: 5427-5437.

MCNALLY, Sean; ROCHE, Jason; CATON, Simon. Predicting the price of Bitcoin using Machine Learning. In: 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP). IEEE, 2018. p. 339-343.

SIN, Edwin; WANG, Lipo. Bitcoin price prediction using ensembles of neural networks. In: 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). IEEE, 2017. p. 666-671.

MATTA, Martina; LUNESU, Ilaria; MARCHESI, Michele. Bitcoin Spread Prediction Using Social and Web Search Media. In: UMAP Workshops. 2015. p. 1-10.

GARCIA, David; SCHWEITZER, Frank. Social signals and algorithmic trading of Bitcoin. Royal Society open science, 2015, 2.9: 150288.

S. Galeshchuk, O. Vasylchyshyn and A. Krysovatyy, “Bitcoin Response to Twitter Sentiments”, in Proc. ICTERI, Kyiv, Ukraine, 2018, pp. 160-168.

J. B. Ramos. “¿Podemos comerciar Bitcoin usando análisis de sentimiento sobre Twitter?”, Trabajo Fin de Grado, Universidad Pontificia de Comillas, Madrid, Spain, 2019.

A. Aggarwal, I. Gupta, N. Garg and A. Goel, "Deep Learning Approach to Determine the Impact of Socio Economic Factors on Bitcoin Price Prediction," 2019 Twelfth International Conference on Contemporary Computing (IC3), Noida, India, 2019, pp. 1-5, doi: 10.1109/IC3.2019.8844928.

Loria, S. (2018). Textblob Documentation (pp. 1-73). Technical report.

A. Karpathy,G. Toderici, S. Shetty, T. Leung, R. Sukthankar, L. Fei-Fei, “Large-scale Video Classification with Convolutional Neural Networks”, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014.

A. Karpathy, L. Fei-Fei, “Deep Visual-Semantic Alignments for Generating Image Descriptions”, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015,pp. 3128-3137.

Andrej Karpathy, Connecting Images and Natural Language, PhD thesis, Stanford University, 2016.

Jiajun Sun, Jing Wang, Ting-chun Yeh, Video Understanding: From Video Classification to Captioning, Stanford University, 2017.

Y.LeCun, Y.Bengio, G.Hinton, "Deep learning", 2015.

B.Ginzburg, "Introduction: Convolutional Neural Networks for Visual Recognition", Intel, 2013

Kilimci, Z. H., & Akyokus, S. (2018). Deep Learning-and Word Embedding-Based Heterogeneous Classifier Ensembles for Text Classification. Complexity, 2018.

Zuxuan Wu, Ting Yao, Yanwei Fu, Yu-Gang Jiang, Deep Learning for Video Classication and Captioning, Fudan University, Microsoft Research Asia, University of Maryland, 2016.

Zuxuan Wu, Ting Yao, Yanwei Fu, Yu-Gang Jiang, Deep Learning for Video Classification and Captioning, University of Maryland, College Park, Microsoft Research Asia, Fudan University, 2018.

Z. C. Lipton, J. Berkowitz, and C. Elkan, “A Critical Review of Recurrent Neural Networks for Sequence Learning,” May 2015.

J. L. Elman, “Finding structure in time,” Cogn. Sci., 1990.

Z. C. Lipton, J. Berkowitz, and C. Elkan, “A Critical Review of Recurrent Neural Networks for Sequence Learning,” May 2015.

J. L. Elman, “Finding structure in time,” Cogn. Sci., 1990.

K. Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A Search Space Odyssey,” IEEE Trans. Neural Networks Learn. Syst., 2017.

D. Kent and F. M. Salem, “Performance of Three Slim Variants of The Long Short-Term Memory {(LSTM)} Layer,” CoRR, vol. abs/1901.00525, 2019.

Q. V. Le and T. Mikolov, “Distributed Representations of Sentences and Documents,” vol. 32, 2014.

T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient Estimation of Word Representations in Vector Space,” pp. 1–12, 2013.

T. Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality arXiv : 1310 . 4546v1 [ cs . CL ] 16 Oct 2013,” Adv. Neural Inf. Process. Syst., 2013.

J. Pennington, R. Socher, and C. Manning, “Glove: Global Vectors for Word Representation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.

A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov, “Bag of Tricks for Efficient Text Classification,” 2016.

T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, and A. Joulin, “Advances in Pre-Training Distributed Word Representations”, arXiv:1712.09405 , 2017.

Kilimci ZH, Akyokus S. N-Gram Pattern Recognition using MultivariateBernoulli Model with Smoothing Methods for Text Classification. 24th IEEE Signal Processing and Communications Applications Conference; 2016; Zonguldak, Turkey.

McCallum A, Nigam KA. Comparison of Event Models for Naive Bayes Text Classification. In: AAAI-98 Workshop on Learning for Text Categorization; 1998; Wisconsin, USA: pp. 41-48.

Kilimci, Z. H., & Omurca, S. I. (2018). The Impact of Enhanced Space Forests with Classifier Ensembles on Biomedical Dataset Classification. International Journal of Intelligent Systems and Applications in Engineering, 6(2), 144-150.

Rennie JDM, Shih L, Teevan J, Karger DR. Tackling the Poor Assumptions of Naive Bayes Text Classifiers. In: 20th International Conference on Machine Learning; 2003; Washington, USA: pp. 616-623.

Kilimci ZH, Ganiz MC. Evaluation of classification models for language processing. In: 10th International Symposium on INnovations in Intelligent SysTems and Applications; 2015; Madrid, Spain: pp. 1-8.

Amasyalı MF, Ersoy OK. Classifier Ensembles with the Extended Space Forest. IEEE Transactions on Knowledge and Data Engineering 2013; 26: 549-562.

Kilimci, Z. H., & Omurca, S. İ. (2018). Extended feature spaces based classifier ensembles for sentiment analysis of short texts. Information Technology and Control, Vol:47, no:3.

Adnan MN, Islam MZ, Kwan PWH. Extended Space Decision Tree. 13th International Conference on Machine Learning and Cybernetics; 2014; Lanzhou, China: pp. 219-230.

Kilimci, Z. H., & Akyokuş, S. (2019, July). The Analysis of Text Categorization Represented With Word Embeddings Using Homogeneous Classifiers. In 2019 IEEE International Symposium on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 1-6). IEEE.

Stenqvist, Evita, and Jacob Lönnö. "Predicting Bitcoin price fluctuation with Twitter sentiment analysis.", INDEGREE PROJECT TECHNOLOGY,FIRST CYCLE, 15 CREDITS,STOCKHOLM SWEDEN, 2017.

Published
2020-06-26
How to Cite
[1]
Z. Kilimci, “Sentiment Analysis Based Direction Prediction in Bitcoin using Deep Learning Algorithms and Word Embedding Models”, IJISAE, vol. 8, no. 2, pp. 60-65, Jun. 2020.
Section
Research Article