Balanced Prediction Based Dynamic Resource Allocation Model for Online Big Data Streams using Historical Data
Keywords:
Streaming Big Data, Historical Data, Dynamic Resource Allocation, Big Data, Minimum Delay, Clustering.Abstract
For cloud computing service providers, one of the largest problems is to maintain good service standards for resource allocation and distribution. The multitenant use of cloud resources with the help of cloud services is one of the most essential utilities in the current period of the technological world. End-user resources should be available with minimal administration and an efficient resource allocation strategy must be developed to avoid scenarios of overprovisioning or under provisioning of resources for handling big data streams. Clients want to keep their costs down, while cloud providers want to get the most out of their existing infrastructure while minimising the need for any new upgrades and provide service with minimum delay. Resource providers can take advantage of the elastic infrastructure provided by cloud computing to obtain streaming capabilities that precisely match demand of users. The amount of cloud resources allocated to users is billed to them. Cloud resource selection has been pioneered by the growing requirement to extract knowledge from massive data streams. The existing methods of resource allocation are dependent on the properties of the data themselves. Despite this, due of the random nature of data generation, it is impossible to predict the features of data in massive data streams. This presents a challenge in selecting and assigning resources to the stream of large data. This work presents a system that anticipates data qualities such as volume, velocity, variety, variability, and truthfulness in order to go in that direction. The proposed model considers weather forecasting data and classifies it as multiple tasks and resource allocation is performed for big data processing. The weather forecasting historical data is used by a set of server groups to assign distinct users jobs to the most trustworthy dynamic resources with less delay.This work presents an effective Balanced Prediction based Resource Allocation for Weather Streaming Data processing using Metadata (BPRA-WSD-MD). The proposed model takes weather forecasting streaming data as input and then divides the data into multiple jobs based on the historical weather report and performs the job execution done successfully by accurate resource allocation model. The proposed model is compared with the traditional models and the results represent that the proposed model performance levels in resource allocation is accurate.
Downloads
References
Ray, B., Saha, A., Khatua, S, Roy, S.: Proactive fault-tolerance technique to enhance reliability of cloud service in cloud federation environment. IEEE Transactions on Cloud Computing (2020)
Maurya, A.K., Modi, K., Kumar, V., Naik, N.S., Tripathi, A.K.: Energy-aware scheduling using slack reclamation for cluster systems. Clust. Comput. 23(2), 911–923 (2020)
Nayak, S.C., Tripathy, C.: Deadline sensitive lease scheduling in cloud computing environment using ahp. J. King Saud Univ. Comput. Inf. Sci. 30(2), 152–163 (2018)
Ray, B.K., Saha, A., Khatua, S., Roy, S.: Toward maximization of profit and quality of cloud federation: solution to cloud federation formation problem. J. Supercomput. 75(2), 885–929 (2019)
Tarafdar, A., Debnath, M., Khatua, S., Das, R.K.: Energy and quality of service-aware virtual machine consolidation in a cloud data center. J. Supercomput., 1–32 (2020)
Peng, Z., Lin, J., Cui, D., Li, Q., He, J.: A multi-objective trade-off framework for cloud resource scheduling based on the deep q-network algorithm. Clust. Comput. 23, 2753–2767 (2020)
Ashraf, A., Porres, I.: Multi-objective dynamic virtual machine consolidation in the cloud using ant colony system. Int. J. Parallel Emergent Distrib. Syst. 33(1), 103–120 (2018)
Fatima, A., Javaid, N., Anjum Butt, A., Sultana, T., Hussain, W., Bilal, M., Akbar, M., Ilahi, M., et al.: An enhanced multi-objective gray wolf optimization for virtual machine placement in cloud data centers. Electronics 8(2), 218 (2019)
Jia, R., Yang, Y., Grundy, J., Keung, J., Li, H.: A deadline constrained preemptive scheduler using queuing systems for multi-tenancy clouds. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 63–67. IEEE (2019)
Khodak, M., Zheng, L., Lan, A.S., Joe-Wong, C., Chiang, M.: Learning cloud dynamics to optimize spot instance bidding strategies. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 2762–2770. IEEE (2018)
Yin, L., Luo, J., Luo, H.: Tasks scheduling and resource allocation in fog computing based on containers for smart manufacturing. IEEE Trans. Industr. Inf. 14(10), 4712–4721 (2018)
S. Agarwal, F. Malandrino, C. F. Chiasserini and S. De, "VNF Placement and Resource Allocation for the Support of Vertical Services in 5G Networks", IEEE/ACM Transactions on Networking, vol. 27, no. 1, pp. 433-446, Feb. 2019.
L. Yala, P. A. Frangoudis, G. Lucarelli and A. Ksentini, "Cost and Availability Aware Resource Allocation and Virtual Function Placement for CDNaaS Provision", IEEE Tran. on Network and Service Management, vol. 15, no. 4, pp. 1334-1348, Dec. 2018.
Lu, L., Yu, J., Zhu, Y., Li, M.: A double auction mechanism to bridge users’ task requirements and providers’ resources in two-sided cloud markets. IEEE Trans. Parallel Distrib. Syst.. 29(4), 720–733 (2018)
Zhang, J., Yang, X., Xie, N., Zhang, X., Vasilakos, A.V., Li, W.: An online auction mechanism for time-varying multidimensional resource allocation in clouds. Future Gener. Comput. Syst. 111, 27–38 (2020)
Middya, A.I., Ray, B., Roy, S.: Auction based resource allocation mechanism in federated cloud environment: TARA. IEEE Transactions on Services Computing (2019)
Patel, Y.S., Nighojkar, A., Misra, R.: Truthful double auction based vm allocation for revenue-energy trade-off in cloud data centers. In: Proceedings of the 2019 National Conference on Communications (NCC), Bangalore, India, pp. 1–6 (2019)
Chen, J.X. Lin, Y. Ma et al., Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. Sci. China Inf. Sci. 62(11), 1–3 (2019)
J. Chen, Y. Wang, A resource request prediction method based on EEMD in cloud computing. Proc. Comput. Sci. 131, 116–123 (2018)
J. Chen, Y. Wang, A hybrid method for short-term host utilization prediction in cloud computing. J. Electr. Comput. Eng. 2782349, 1–14 (2019)
D. Shen, Research on application-aware resource management for heterogeneous big data workloads in cloud environment. Dongnan University, 2018.
X. Chen, J. X. Lin, B. Lin, T. Xiang, Y. Zhang and G. Huang, Self-learning and self-adaptive resource allocation for cloud-based software services. Concurrency Comput. Pract. Exp., 31(23), e4463 (2019).
K. Gurleen, B. Anju, A survey of prediction-based resource scheduling techniques for physics-based scientific applications, Mod. Phys. Lett. B, 32(25), 1850295(2018).
Y.J. Laili, S.S. Lin, D.Y. Tang, Multi-phase integrated scheduling of hybrid tasks in cloud manufacturing environment. Robot. Comput. Integr. Manuf. 61, 101850 (2020)
K. Reihaneh, S.E. Faramarz, N. Naser, M. Mehran, ATSDS: adaptive two-stage deadline-constrained workflow scheduling considering run-time circumstances in cloud computing environments. J. Supercomput. 73(6), 2430–2455 (2017)
K. Kavitha, S. C. Sharma, Performance analysis of ACO-based improved virtual machine allocation in cloud for IoT-enabled healthcare. Concurr. Comput. Pract. Exp., e5613 (2019).
J. Vahidi, M. Rahmati, in IEEE 5th Conference on Knowledge Based Engineering and Innovation (KBEI). Optimization of resource allocation in cloud computing by grasshopper optimization algorithm, pp. 839–844 (2019).
U. Rugwiro, C.H. Gu, W.C. Ding, Task scheduling and resource allocation based on ant-colony optimization and deep reinforcement learning. J. Internet Technol. 20(5), 1463–1475 (2019)
S. Shenoy, D. Gorinevsky, N. Laptev, Probabilistic Modelling of Computing Request for Service Level Agreement. IEEE Trans. Serv. Comput. 12(6), 987–993 (2019)
Z.H. Liu, Z.J. Wang, C. Yang, Multi-objective resource optimization scheduling based on iterative double auction in cloud manufacturing. Adv. Manuf. 7(4), 374–388 (2019)
A. Motlagh, A. Movaghar, A. M. Rahmani, Task scheduling mechanism in cloud computing: a systematic review. Int. J. Commun. Syst. e4302 (2019).
M. Kumar, S.C. Sharma, A. Goel, S.P. Singh, A comprehensive survey for scheduling techniques in cloud computing. J. Netw. Comput. Appl. 143, 1–33 (2019)
N. D. Vahed, M. Ghobaei-Arani, A. Souri, Multiobjective virtual machine placement mechanisms using nature-inspired metaheuristic algorithms in cloud environments: a comprehensive review. Int. J. Commun. Syst. 32(14), e4068 (2019).
F. Sheikholeslami, N. J. Navimipour, Auction-based resource allocation mechanisms in the cloud environments: a review of the literature and reflection on future challenges. Concurr. Computat. Pract. Exp., 30(16), e4456 (2018).
G. Natesan, A. Chokkalingam, An improved grey wolf optimization algorithm based task scheduling in cloud computing environment. Int. Arab J. Inf. Technol. 17(1), 73–81 (2020)
M. A. Reddy, K. Ravindranath, Virtual machine placement using JAYA optimization algorithm. Appl. Artif. Intell. https://doi.org/10.1080/08839514.2019.1689714.
S. Souravlas, S. Katsavounis, Scheduling fair resource allocation policies for cloud computing through flow control. Electronics 8(11), 1348 (2019).
L. Guo, P. Du, A. Razaque, et al. IEEE 2018 Fifth international conference on software defined systems (SDS). Energy saving and maximize utilization cloud resources allocation via online multi-dimensional vector bin packing (2018), pp. 160–165.
N. Gul, I. A. Khan, S. Mustafa, o. Khalid, A. U. R. Khan, CPU-RAM-based energy-efficient resource allocation in clouds. J. Supercomput. 75(11), 7606–7624 (2019).
R.L. Sri, N. Balaji, An empirical model of adaptive cloud resource provisioning with speculation. Soft. Comput. 23(21), 10983–10999 (2019)
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Vijaya Kumar Chandarapu, Madhavi Kasa
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
