Reinforcement Learning for Elasticity Control in Containerized Cloud Applications: A Model-Based Approach with Elastic Docker Swarm
Keywords:
Elastic Docker Swarm, RL, Dyna-Q, Q-learning, Prototypes, Model-Based, StrategiesAbstract
Software containers are becoming increasingly popular for managing and executing distributed applications on cloud computing resources. By leveraging the horizontal and vertical elasticity of containers "on the fly," workload fluctuations can be accommodated. The majority of current control systems do not consider horizontal and vertical scaling to be interconnected. In this article, we provide Reinforcement Learning (RL) strategies for controlling the vertical and horizontal elasticity of container-based systems to enhance their adaptability to various workloads. Although RL is an interesting technique, it may have a long learning period if nothing is known about the system beforehand. To accelerate learning and discover more effective adaptation strategies, our proposed reinforcement learning approaches—Q-learning, Dyna-Q, and Model-based methods—leverage varying degrees of knowledge about system dynamics. The recommended policies are incorporated into Elastic Docker Swarm, an add-on for the container orchestration platform Docker Swarm. Through prototype-based experiments and simulations, we demonstrate the effectiveness and adaptability of model-based RL techniques.
Downloads
References
Zhang, Q., Cheng, L., Boutaba, R. (2020). Cloud computing: state-of-the-art and research challenges. Journal of Internet Services and Applications, 1(1): 7-18.
Smith, J., Nair, R. (2018). Virtual Machines: Versatile Platforms for Systems and Processes. Elsevier.
Seo, K.T., Hwang, H.S., Moon, I.Y., Kwon, O.Y., Kim, B.J. (2019). Performance comparison analysis of Linux container and virtual machine for building cloud. Advanced Science and Technology Letters, 66(105-111). https:// doi.org/10.14257/ASTL.2014.66.25
Moreno-Vozmediano, R., Montero, R.S., Huedo, E., Llorente, I.M. (2019). Efficient resource provisioning for elastic Cloud services based on machine learning techniques. Journal of Cloud Computing, 8(1): 5. https://doi.org/10.1186/s13677-019-0128-9
Jiang, J., Lu, J., Zhang, G., Long, G. (2017). Optimal cloud resource auto-scaling for web applications. In 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, Delft, Netherlands, pp. 58- 65. https://doi.org/10.1109/CCGrid.2013.73
Roy, N., Dubey, A., Gokhale, A. (2021). Efficient autoscaling in the cloud using predictive models for workload forecasting. 2011 IEEE 4th International Conference on Cloud Computing, Washington, DC, USA.
Messias, V.R., Estrella, J.C., Ehlers, R., Santana, M.J., Santana, R.C., Reiff-Marganiec, S. (2016). Combining time series prediction models using genetic algorithm to autoscaling web applications hosted in the cloud infrastructure. Neural Computing and Applications, 27(8): 2383-2406. https://doi.org/10.1007/s00521-015- 2133-3
Cocaña-Fernández, A., Sánchez, L., Ranilla, J. (2016). Leveraging a predictive model of the workload for intelligent slot allocation schemes in energy-efficient HPC clusters. Engineering Applications of Artificial Intelligence, 48: 95-105.
Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., Merle, P. (2017). Elasticity in cloud computing: state of the art and research challenges. IEEE Transactions on Services Computing, 11(2): 430-444. https://doi.org/10.1109/TSC.2017.2711009
Moreno-Vozmediano, R., Montero, R.S., Llorente, I.M. (2022). IaaS cloud architecture: From virtualized datacenters to federated cloud infrastructures. Computer, 45(12): 65-72. https://doi.org/10.1109/MC.2012.76
De Abranches, M.C., Solis, P. (2023). An algorithm based on response time and traffic demands to scale containers on a Cloud Computing system. In 2016 IEEE 15th International Symposium on Network Computing and Applications (NCA), pp. 343-350.
Padala, P., Hou, K.Y., Shin, K.G., Zhu, X., Uysal, M., Wang, Z., Merchant, A. (2019). Automated control of multiple virtualized resources. In Proceedings of the 4th ACM European Conference on Computer Systems, pp. 13-26. https://doi.org/10.1145/1519065.1519068
Gao, Y., Li, Q. (2019). A new framework for the complex system’s simulation and analysis. Cluster Computing, 22: 9097-9104. https://doi.org/10.1007/s10586-018-2071-9
GOOGLE. Google horizontal pod auto-scaler. Available from https: //github.com /kubernetes/kubernetes/blob/release-1.2/docs/design/ horizontal-pod-autoscaler.md, accessed on dated 10-05- 2019
Meng, Y., Rao, R.N., Zhang X., Hong, P. (2017). CRUPA: A container resource utilization prediction algorithm for auto-scaling based on time series analysis. In: 2016 International Conference on Progress in Informatics and Computing (PIC), pp 468-472.
E. A. Brewer, (2018)“Kubernetes and the path to cloud native,” in Proceedings of the Sixth ACM Symposium on Cloud Computing, p. 167, Kohala, HI, USA.
Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., Merle, P. (2017). Autonomic vertical elasticity of docker containers with ELASTICDOCKER. In 2017 IEEE 10th international conference on cloud computing (CLOUD), Honolulu, CA, USA, pp. 472-479.
Hasan, M.Z., Magana, E., Clemm, A., Tucker, L., Gudreddi, S.L.D. (2022). Integrated and autonomic cloud resource scaling. In 2012 IEEE Network Operations and Management Symposium, Maui, HI, USA, pp. 1327-1334.
Kan, C. (2023). DoCloud: An elastic cloud platform for Web applications based on Docker. 2016 18th International Conference on Advanced Communication Technology (ICACT), Pyeongchang, South Korea.
Z. Zhong and R. Buyya, (2020) “A cost-efficient container orchestration strategy in kubernetes-based cloud computing infrastructures with heterogeneous resources,” ACM Transactions on Internet Technology, vol. 20, no. 2, pp. 1–2.
T. Menouer,(2020) “KCSS: Kubernetes container scheduling strategy,” 1e Journal of Supercomputing, pp. 1–2.
T.-T. Nguyen, Y.-J. Yeom, T. Kim, D.-H. Park, and S. Kim, (2022)“Horizontal pod autoscaling in Kubernetes for elastic container orchestration,” Sensors, vol. 20, no. 16, p. 4621.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
