Optimizing Resource Allocation in Big Data Scheduling Architectures Using a Tuned Firefly Algorithm
Keywords:
Resource Allocation, Swarm Intelligence, Quality of Service, Service Level Agreements, Big Data Scheduling, Optimization, Efficiency, Load Factors, Computational Resources, Dynamic WorkloadsAbstract
In contemporary research architectures, the efficient allocation of computational resources is paramount to meet the dynamic and diverse demands of tasks and applications. Traditional resource allocation policies often struggle with adaptability to varying workloads and underutilize historical allocation data. To address these limitations and enhance Quality of Service (QoS) in research environments, a rank-based data node allocation system is imperative. This research introduces a novel firefly-based Swarm Intelligence (SI) algorithm, meticulously tuned to strike a balance between resource overutilization and underutilization while upholding essential Service Level Agreements (SLAs). The contributions of this work encompass algorithm design, the delicate equilibrium between resource usage and SLA adherence, the establishment of a robust evaluation framework, and systematic comparisons with state-of-the-art resource allocation methods. Furthermore, this study takes into account different load factors, providing a comprehensive analysis of resource allocation efficiency across varying workloads. The algorithm's performance is evaluated based on QoS metrics, including power consumption and SLA adherence, and compared with existing allocation methods. The results underscore the algorithm's superiority in enhancing resource allocation efficiency and SLA adherence in Big Data Scheduling Architectures.
Downloads
References
S. Shadroo and A. M. Rahmani, “Systematic survey of big data and data mining in internet of things,” Computer Networks, vol. 139, pp. 19–47, Jul. 2018, doi: 10.1016/J.COMNET.2018.04.001.
C. L. Philip Chen and C. Y. Zhang, “Data-intensive applications, challenges, techniques and technologies: A survey on Big Data,” Information Sciences, vol. 275, pp. 314–347, Aug. 2014, doi: 10.1016/J.INS.2014.01.015.
Y. Hajjaji, W. Boulila, I. R. Farah, I. Romdhani, and A. Hussain, “Big data and IoT-based applications in smart environments: A systematic review,” Computer Science Review, vol. 39, p. 100318, Feb. 2021, doi: 10.1016/J.COSREV.2020.100318.
Y. Jiang, Z. Huang, and D. H. K. Tsang, “Towards Max-Min Fair Resource Allocation for Stream Big Data Analytics in Shared Clouds,” IEEE Transactions on Big Data, vol. 4, no. 1, pp. 130–137, Dec. 2016, doi: 10.1109/TBDATA.2016.2638860.
R. Buyya, A. Beloglazov, and J. Abawajy, “Energy-Efficient Management of Data Center Resources for Cloud Computing: A Vision, Architectural Elements, and Open Challenges,” Jun. 2010, doi: 10.48550/arxiv.1006.0308.
W. T. Wu, W. W. Lin, C. H. Hsu, and L. G. He, “Energy-efficient hadoop for big data analytics and computing: A systematic review and research insights,” Future Generation Computer Systems, vol. 86, pp. 1351–1367, Sep. 2018, doi: 10.1016/J.FUTURE.2017.11.010.
M. Senthilkumar and P. Ilango, “Energy aware task scheduling using hybrid firefly - GA in big data,” International Journal of Advanced Intelligence Paradigms, vol. 16, no. 2, pp. 99–112, 2020, doi: 10.1504/IJAIP.2020.107008.
C. Li, Y. P. Wang, Y. Chen, and Y. Luo, “Energy-efficient fault-tolerant replica management policy with deadline and budget constraints in edge-cloud environment,” Journal of Network and Computer Applications, vol. 143, pp. 152–166, Oct. 2019, doi: 10.1016/J.JNCA.2019.04.018.
A. Tzanetos and G. Dounias, “A Comprehensive Survey on the Applications of Swarm Intelligence and Bio-Inspired Evolutionary Strategies,” Machine Learning Paradigms, pp. 337–378, 2020, doi: 10.1007/978-3-030-49724-8_15.
L. Abualigah et al., “Advances in Meta-Heuristic Optimization Algorithms in Big Data Text Clustering,” Electronics 2021, Vol. 10, Page 101, vol. 10, no. 2, p. 101, Jan. 2021, doi: 10.3390/ELECTRONICS10020101.
B. H. Nguyen, B. Xue, and M. Zhang, “A survey on swarm intelligence approaches to feature selection in data mining,” Swarm and Evolutionary Computation, vol. 54, p. 100663, May 2020, doi: 10.1016/J.SWEVO.2020.100663.
S. Kumar and S. K. Goyal, “Swarm Intelligence Based Data Selection Mechanism for Reputation Generation in Social Cloud,” 2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing, COM-IT-CON 2022, pp. 583–588, 2022, doi: 10.1109/COM-IT-CON54601.2022.9850947.
L. Brezočnik, I. Fister, and V. Podgorelec, “Swarm Intelligence Algorithms for Feature Selection: A Review,” Applied Sciences 2018, Vol. 8, Page 1521, vol. 8, no. 9, p. 1521, Sep. 2018, doi: 10.3390/APP8091521.
T. Jayaraj, “Process Optimization of Big-Data Cloud Centre Using Nature Inspired Firefly Algorithm and K-Means Clustering,” Article in International Journal of Innovative Technology and Exploring Engineering, vol. 8, no. 12, pp. 2278–3075, 2019, doi: 10.35940/ijitee.L2490.1081219..
G. Rjoub, J. Bentahar, and O. A. Wahab, “BigTrustScheduling: Trust-aware big data task scheduling approach in cloud computing environments,” Future Generation Computer Systems, vol. 110, pp. 1079–1097, 2020, doi: 10.1016/j.future.2019.11.019.
K. Manivannane and M. Chidambaram, “A Cloud Resource Scheduling Framework for big data stream and analytics in Cloud Environment,” Proceedings of the 4th International Conference on Communication and Electronics Systems, ICCES 2019, pp. 1638–1642, Jul. 2019, doi: 10.1109/ICCES45898.2019.9002225.
H. GIBET TANI and C. EL AMRANI, “Smarter Round Robin Scheduling Algorithm for Cloud Computing and Big Data,” Journal of Data Mining & Digital Humanities, Jan. 2018, doi: 10.46298/jdmdh.3104.
H. Wang et al., “A hybrid multi-objective firefly algorithm for big data optimization,” Applied Soft Computing, vol. 69, pp. 806–815, Aug. 2018, doi: 10.1016/J.ASOC.2017.06.029.
T. Jayaraj, “Process Optimization of Big-Data Cloud Centre Using Nature Inspired Firefly Algorithm and K-Means Clustering,” Article in International Journal of Innovative Technology and Exploring Engineering, vol. 8, no. 12, pp. 2278–3075, 2019, doi: 10.35940/ijitee.L2490.1081219.
M. T. Alharbi, Y. Qawqzeh, A. Jaradat, K. Nazim, and A. Sattar, “A review of swarm intelligence algorithms deployment for scheduling and optimization in cloud computing environments,” PeerJ Computer Science, vol. 7, p. e696, Aug. 2021, doi: 10.7717/PEERJ-CS.696.
S. Talwani et al., “Machine-Learning-Based Approach for Virtual Machine Allocation and Migration,” Electronics, vol. 11, no. 19, p. 3249, 2022.
O. A. Oduwole, S. A. Akinboro, O. G. Lala, and S. O. Olabiyisi, “An Enhanced Load Balancing Technique for Big-data Cloud Computing Environments,” Transactions of the Royal Society of South Africa, vol. 77, no. 3, pp. 219–236, Sep. 2022, doi: 10.1080/0035919X.2022.2160389.
X. Wang, X. Hu, W. Fan, and R. Wang, “Efficient data persistence and data division for distributed computing in cloud data center networks,” The Journal of Supercomputing, vol. 79, no. 14, pp. 16300–16327, Sep. 2023, doi: 10.1007/S11227-023-05276-2.
A. Kaur et al., “Algorithmic Approach to Virtual Machine Migration in Cloud Computing with Updated SESA Algorithm,” Sensors (Basel, Switzerland), vol. 23, no. 13, p. 6117, Jul. 2023, doi: 10.3390/S23136117.
S. Mangalampalli, G. R. Karri, and A. A. Elngar, “An Efficient Trust-Aware Task Scheduling Algorithm in Cloud Computing Using Firefly Optimization,” Sensors 2023, Vol. 23, Page 1384, vol. 23, no. 3, p. 1384, Jan. 2023, doi: 10.3390/S23031384.
K. Hassan, N. Javaid, F. Zafar, S. Rehman, M. Zahid, and S. Rasheed, “A Cloud Fog Based Framework for Efficient Resource Allocation Using Firefly Algorithm,” Lecture Notes on Data Engineering and Communications Technologies, vol. 25, pp. 431–443, 2019, doi: 10.1007/978-3-030-02613-4_38/COVER.
N. Bacanin, M. Zivkovic, T. Bezdan, K. Venkatachalam, and M. Abouhawwash, “Modified firefly algorithm for workflow scheduling in cloud-edge environment,” Neural Computing and Applications, vol. 34, no. 11, pp. 9043–9068, Jun. 2022, doi: 10.1007/S00521-022-06925-Y/FIGURES/9.
M. A. Elmagzoub, D. Syed, A. Shaikh, N. Islam, A. Alghamdi, and S. Rizwan, “A Survey of Swarm Intelligence Based Load Balancing Techniques in Cloud Computing Environment,” Electronics vol. 10, no. 21, p. 2718, Nov. 2021, doi: 10.3390/ELECTRONICS10212718.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
