Distributed Algorithms for Large-Scale Computing in Cloud Environments: A Review of Parallel and Distributed Processing
Keywords:
Cloud Computing, Distributed Algorithms, Large-Scale computing, Parallel Processing, Distributed ProcessingAbstract
The popularity of cloud computing and large-scale distributed systems is rapidly increasing because of the variety of service models and advantages they offer as well as the necessity of individuals and organizations to access their resources easily and efficiently, in addition to the need for more reliable and robust systems. For these reasons, many distributed algorithms have been designed to facilitate the coordination and interconnection among the distributed computational elements to work together in parallel to achieve a common goal. These algorithms are related to various aspects such as consensus, load balancing, scheduling, communication, leader selection and fault tolerance. Many researches have been carried out to investigate and improve the performance of these distributed algorithms. Therefore, this paper studies and compares a variety of research works that has been performed in distributed algorithms for large-scale cloud computing.
Downloads
References
B. Varghese and R. Buyya, “Next generation cloud computing: New trends and research directions,” Future Generation Computer Systems, vol. 79, pp. 849–861, 2018.
J. Chinna and K. Kavitha, “A study on Large Scale Graph Processing Frameworks Performance for Cloud Operations in different nodes.”
F. Li, “Execution Feature Extraction and Prediction for Large-Scale Graph Processing Applications,” in 2019 Seventh International Conference on Advanced Cloud and Big Data (CBD), 2019, pp. 84–89.
M. Birje, P. S. Challagidad, M. Tapale, and R. Goudar, “Security issues and countermeasures in cloud computing,” International Journal of Applied Engineering Research, vol. 10, no. 86, pp. 71–5, 2015.
P. Kumari and P. Kaur, “A survey of fault tolerance in cloud computing,” Journal of King Saud University-Computer and Information Sciences, vol. 33, no. 10, pp. 1159–1176, 2021.
P. Y. Abdullah, S. Zeebaree, H. M. Shukur, and K. Jacksi, “HRM system using cloud computing for Small and Medium Enterprises (SMEs),” Technology Reports of Kansai University, vol. 62, no. 04, p. 04, 2020.
J. Zhang, H. Huang, and X. Wang, “Resource provision algorithms in cloud computing: A survey,” Journal of network and computer applications, vol. 64, pp. 23–42, 2016.
D. A. Shafiq, N. Z. Jhanjhi, A. Abdullah, and M. A. Alzain, “A load balancing algorithm for the data centres to optimize cloud computing applications,” IEEE Access, vol. 9, pp. 41731–41744, 2021.
S. Heidari and R. Buyya, “A cost-efficient auto-scaling algorithm for large-scale graph processing in cloud environments with heterogeneous resources,” IEEE Transactions on Software Engineering, vol. 47, no. 8, pp. 1729–1741, 2019.
S. R. Zeebaree, H. M. Shukur, L. M. Haji, R. R. Zebari, K. Jacksi, and S. M. Abas, “Characteristics and analysis of hadoop distributed systems,” Technology Reports of Kansai University, vol. 62, no. 4, pp. 1555–1564, 2020.
P. Y. Abdullah, S. Zeebaree, K. Jacksi, and R. R. Zeabri, “An hrm system for small and medium enterprises (sme) s based on cloud computing technology,” International Journal of Research-GRANTHAALAYAH, vol. 8, no. 8, pp. 56–64, 2020.
A. Rehman, R. L. Aguiar, and J. P. Barraca, “Fault-tolerance in the scope of cloud computing,” IEEE Access, vol. 10, pp. 63422–63441, 2022.
M. M. A. Baig, “An Evaluation of Major Fault Tolerance Techniques Used on High Performance Computing (HPC) Applications,” International Journal of Intelligent Systems and Applications in Engineering, vol. 11, no. 3s, pp. 320–328, 2023.
S. Chand and Y. A. Liu, “Brief Announcement: What’s Live? Understanding Distributed Consensus,” in Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing, 2021, pp. 565–568.
S. S. Priya and T. Rajendran, “Improved round-robin rule learning for optimal load balancing in distributed cloud systems,” International Journal of System of Systems Engineering, vol. 13, no. 1, pp. 83–99, 2023.
J. Nikoli’c, N. Jubatyrov, and E. Pournaras, “Self-healing dilemmas in distributed systems: Fault correction vs. fault tolerance,” IEEE Transactions on Network and Service Management, vol. 18, no. 3, pp. 2728–2741, 2021.
D. Tomar and P. Tomar, “Integration of cloud computing and big data technology for smart generation,” in 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence), 2018, pp. 1–6.
Z. N. Rashid, S. R. Zeebaree, M. A. Sadeeq, R. R. Zebari, H. M. Shukur, and A. Alkhayyat, “Cloud-based Parallel Computing System Via Single-Client Multi-Hash Single-Server Multi-Thread,” in 2021 International Conference on Advance of Sustainable Engineering and its Application (ICASEA), 2021, pp. 59–64.
S. Zeebaree and others, “DES encryption and decryption algorithm implementation based on FPGA,” Indones. J. Electr. Eng. Comput. Sci, vol. 18, no. 2, pp. 774–781, 2020.
Y. Afek, G. Giladi, and B. Patt-Shamir, “Distributed Computing with the Cloud,” in Stabilization, Safety, and Security of Distributed Systems: 23rd International Symposium, SSS 2021, Virtual Event, November 17-20, 2021, Proceedings 23, 2021, pp. 1–20.
E. Jonas, Q. Pu, S. Venkataraman, I. Stoica, and B. Recht, “Occupy the cloud: Distributed computing for the 99%,” in Proceedings of the 2017 symposium on cloud computing, 2017, pp. 445–451.
S. R. Zeebaree, A. B. Sallow, B. K. Hussan, and S. M. Ali, “Design and simulation of high-speed parallel/sequential simplified DES code breaking based on FPGA,” in 2019 International Conference on Advanced Science and Engineering (ICOASE), 2019, pp. 76–81.
S. Zeebaree, R. R. Zebari, K. Jacksi, and D. A. Hasan, “Security approaches for integrated enterprise systems performance: A Review,” Int. J. Sci. Technol. Res, vol. 8, no. 12, pp. 2485–2489, 2019.
O. Bystrov, A. Kavceniauskas, and R. Pacevivc, “Cost and Performance Analysis of MPI-Based SaaS on the Private Cloud Infrastructure,” in International Conference on Parallel Processing and Applied Mathematics, 2022, pp. 171–182.
M. Masdari, F. Salehi, M. Jalali, and M. Bidaki, “A survey of PSO-based scheduling algorithms in cloud computing,” Journal of Network and Systems Management, vol. 25, no. 1, pp. 122–158, 2017.
Z. S. Ageed and S. R. Zeebaree, “Distributed Systems Meet Cloud Computing: A Review of Convergence and Integration,” International Journal of Intelligent Systems and Applications in Engineering, vol. 12, no. 11s, pp. 469–490, 2024.
S. Basu, A. Bardhan, K. Gupta, P. Saha, M. Pal, M. Bose, K. Basu, S. Chaudhury, and P. Sarkar, “Cloud computing security challenges & solutions-A survey,” in 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), 2018, pp. 347–356.
H. Malallah, S. R. Zeebaree, R. R. Zebari, M. A. Sadeeq, Z. S. Ageed, I. M. Ibrahim, H. M. Yasin, and K. J. Merceedi, “A comprehensive study of kernel (issues and concepts) in different operating systems,” Asian Journal of Research in Computer Science, vol. 8, no. 3, pp. 16–31, 2021.
H. Shukur, S. Zeebaree, R. Zebari, O. Ahmed, L. Haji, and D. Abdulqader, “Cache coherence protocols in distributed systems,” Journal of Applied Science and Technology Trends, vol. 1, no. 3, pp. 92–97, 2020.
D. A. Shafiq, N. Jhanjhi, and A. Abdullah, “Load balancing techniques in cloud computing environment: A review,” Journal of King Saud University-Computer and Information Sciences, vol. 34, no. 7, pp. 3910–3933, 2022.
D. A. Hasan, B. K. Hussan, S. R. Zeebaree, D. M. Ahmed, O. S. Kareem, and M. A. Sadeeq, “The impact of test case generation methods on the software performance: A review,” International Journal of Science and Business, vol. 5, no. 6, pp. 33–44, 2021.
M. Xu, W. Tian, and R. Buyya, “A survey on load balancing algorithms for virtual machines placement in cloud computing,” Concurrency and Computation: Practice and Experience, vol. 29, no. 12, p. e4123, 2017.
H. M. Zangana and S. R. Zeebaree, “Distributed Systems for Artificial Intelligence in Cloud Computing: A Review of AI-Powered Applications and Services,” International Journal of Informatics, Information System and Computer Engineering (INJIISCOM), vol. 5, no. 1, pp. 1–20, 2024.
S. R. Zeebaree, L. M. Haji, I. Rashid, R. R. Zebari, O. M. Ahmed, K. Jacksi, and H. M. Shukur, “Multicomputer multicore system influence on maximum multi-processes execution time,” TEST Engineering & Management, vol. 83, no. 03, pp. 14921–14931, 2020.
D. M. ABDULQADER, S. R. ZEEBAREE, R. R. ZEBARI, S. A. SALEH, Z. N. RASHID, and M. A. SADEEQ, “SINGLE-THREADING BASED DISTRIBUTED-MULTIPROCESSOR-MACHINES AFFECTING BY DISTRIBUTED-PARALLEL-COMPUTING TECHNOLOGY,” Journal of Duhok University, vol. 26, no. 2, pp. 416–426, 2023.
L. M. Haji, S. R. Zeebaree, O. M. Ahmed, M. A. Sadeeq, H. M. Shukur, and A. Alkhavvat, “Performance Monitoring for Processes and Threads Execution-Controlling,” in 2021 International Conference on Communication & Information Technology (ICICT), 2021, pp. 161–166.
L. M. Haji, S. R. Zeebaree, Z. S. Ageed, O. M. Ahmed, M. A. Sadeeq, and H. M. Shukur, “Performance Monitoring and Controlling of Multicore Shared-Memory Parallel Processing Systems,” in 2022 3rd Information Technology To Enhance e-learning and Other Application (IT-ELA), 2022, pp. 44–48.
Y. Chen, B. Brock, S. Porumbescu, A. Buluc, K. Yelick, and J. Owens, “Atos: A task-parallel GPU scheduler for
K. Zhang, Y. Fang, Y. Zheng, H. Zeng, L. Xu, and W. Wang, “Graphlib: A parallel graph mining library for joint cloud computing,” in 2020 IEEE International Conference on Joint Cloud Computing, 2020, pp. 9–12.
S. Zeebaree and K. Jacksi, “Effects of processes forcing on CPU and total execution-time using multiprocessor shared memory system,” Int. J. Comput. Eng. Res. Trends, vol. 2, no. 4, pp. 275–279, 2015.
Z. N. Rashid, S. R. Zeebaree, R. R. Zebari, S. H. Ahmed, H. M. Shukur, and A. Alkhayyat, “Distributed and Parallel Computing System Using Single-Client Multi-Hash Multi-Server Multi-Thread,” in 2021 1st Babylon International Conference on Information Technology and Science (BICITS), 2021, pp. 222–227.
M. A. Sadeeq and S. R. Zeebaree, “Design and implementation of an energy management system based on distributed IoT,” Computers and Electrical Engineering, vol. 109, p. 108775, 2023.
S. Li, M. A. Maddah-Ali, Q. Yu, and A. S. Avestimehr, “A fundamental tradeoff between computation and communication in distributed computing,” IEEE Transactions on Information Theory, vol. 64, no. 1, pp. 109–128, 2017.
M. A. Sadeeq and S. R. Zeebaree, “DPU-ALDOSKI dataset of Monitoring and Controlling distributed far distances energy consumed system based on Internet of Things,” Data in Brief, vol. 49, p. 109455, 2023.
Y. S. Jghef, S. R. Zeebaree, Z. S. Ageed, and H. M. Shukur, “Performance Measurement of Distributed Systems via Single-Host Parallel Requesting using (Single, Multi and Pool) Threads,” in 2022 3rd Information Technology To Enhance e-learning and Other Application (IT-ELA), 2022, pp. 38–43.
X. Zhu, J. Shi, S. Huang, and B. Zhang, “Consensus-oriented cloud manufacturing based on blockchain technology: An exploratory study,” Pervasive and mobile computing, vol. 62, p. 101113, 2020.
Y. Wang and Y. Chai, “vRaft: accelerating the distributed consensus under virtualized environments,” in Database Systems for Advanced Applications: 26th International Conference, DASFAA 2021, Taipei, Taiwan, April 11-14, 2021, Proceedings, Part I 26, 2021, pp. 53–70.
M. H. Cintuglu and D. Ishchenko, “Real-time asynchronous information processing in distributed power systems control,” IEEE Transactions on Smart Grid, vol. 13, no. 1, pp. 773–782, 2021.
D. Mekonnen, A. Megersa, R. K. Sharma, D. P. Sharma, and others, “Designing a Component-Based Throttled Load Balancing Algorithm for Cloud Data Centers,” Mathematical Problems in Engineering, vol. 2022, 2022.
W. Zhong, C. Yang, W. Liang, J. Cai, L. Chen, J. Liao, and N. Xiong, “Byzantine Fault-Tolerant Consensus Algorithms: A Survey,” Electronics, vol. 12, no. 18, p. 3801, 2023.
R. Guo, Z. Guo, Z. Lin, and W. Jiang, “A hierarchical byzantine fault tolerance consensus protocol for the internet of things,” High-Confidence Computing, p. 100196, 2023.
V. K. Madisetti and S. Panda, “A dynamic leader election algorithm for decentralized networks,” Journal of Transportation Technologies, vol. 11, no. 3, pp. 404–411, 2021.
E. Wang, Y. Barve, A. Gokhale, and H. Sun, “Dynamic Resource Management for Cloud-native Bulk Synchronous Parallel Applications,” in 2023 IEEE 26th International Symposium on Real-Time Distributed Computing (ISORC), 2023, pp. 152–157.
Y.-J. Gong, W.-N. Chen, Z.-H. Zhan, J. Zhang, Y. Li, Q. Zhang, and J.-J. Li, “Distributed evolutionary algorithms and their models: A survey of the state-of-the-art,” Applied Soft Computing, vol. 34, pp. 286–300, 2015.
D. House, H. Kuang, K. Surendran, and P. Chen, “Toward fast and reliable active-active geo-replication for a distributed data caching service in the mobile cloud,” Procedia Computer Science, vol. 191, pp. 119–126, 2021.
Y. Wang, Z. Wang, Y. Chai, and X. Wang, “Rethink the linearizability constraints of raft for distributed key-value stores,” in 2021 IEEE 37th International Conference on Data Engineering (ICDE), 2021, pp. 1877–1882.
A. Charapko, A. Ailijiang, and M. Demirbas, “Pigpaxos: Devouring the communication bottlenecks in distributed consensus,” in Proceedings of the 2021 International Conference on Management of Data, 2021, pp. 235–247.
K. K. Agarwal and H. Kotakula, “Replication Based Fault Tolerance Approach for Cloud,” in International Conference on Distributed Computing and Internet Technology, 2022, pp. 163–169.
Y. Li, L. Qiao, and Z. Lv, “An optimized byzantine fault tolerance algorithm for consortium blockchain,” Peer-to-Peer Networking and Applications, vol. 14, pp. 2826–2839, 2021.
S. Kanwal, Z. Iqbal, A. Irtaza, M. Sajid, S. Manzoor, and N. Ali, “Head node selection algorithm in cloud computing data center,” Mathematical problems in engineering, vol. 2021, pp. 1–12, 2021.
I. Abraham, D. Dolev, and J. Y. Halpern, “Distributed protocols for leader election: A game-theoretic perspective,” ACM Transactions on Economics and Computation (TEAC), vol. 7, no. 1, pp. 1–26, 2019.
S. S. Supase and R. B. Ingle, “Are coordinator election algorithms in distributed systems vulnerable?,” in 2020 11th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2020, pp. 1–5.
A. Katangur, S. Akkaladevi, and S. Vivekanandhan, “Priority Weighted Round Robin Algorithm for Load Balancing in the Cloud,” in 2022 IEEE 7th International Conference on Smart Cloud (SmartCloud), 2022, pp. 230–235.
P. Kijsanayothin, G. Chalumporn, and R. Hewett, “On using MapReduce to scale algorithms for Big Data analytics: a case study,” Journal of Big Data, vol. 6, pp. 1–20, 2019.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
