Automating Machine Learning Workflows with Cloud-Based Pipelines
Keywords:
inefficiency, accomplishing, organization, enhancementAbstract
The paper is aimed at discussing cloud-based pipelines for automating machine learning processes. The paper also discusses how these types of systems overcome fundamental issues, which are associated with ML processes such as, including inefficiency, scalability problems and convolutions of collaborating among other similar systems. Cloud-based pipelines use distributed computation and storage to automate the whole ML pipeline right from data processing to model deploying. The study identifies advantages including more efficient process organization, managing resources as well as better integration of employees. Techniques that have been examined are automated data pipeline creation, large-scale model building and training and methods on service deployment and maintenance. Major findings show that the use of this framework leads to a reduction of the time required for accomplishing ML projects and enhancement in the quality of the models developed, in addition to facilitating effective replication of experiments.
Downloads
References
Alarcon, M.L., Oruche, R., Pandey, A. and Calyam, P., 2022. Cloud-based data pipeline orchestration platform for COVID-19 evidence-based analytics. In Novel AI and Data Science Advancements for Sustainability in the Era of COVID-19 (pp. 159-180). Academic Press.
Bartezzaghi, A., Giurgiu, I., Marchiori, C., Rigotti, M., Sebastian, R. and Malossi, C., 2022, June. Design of a Cloud-Based Data Platform for Standardized Machine Learning Workflows with Applications to Transport Infrastructure. In 2022 IEEE 21st Mediterranean Electrotechnical Conference (MELECON) (pp. 764-769). IEEE.
Bustamante, A.L., Patricio, M.A., Berlanga, A. and Molina, J.M., 2023. Seamless transition from machine learning on the cloud to industrial edge devices with thinger. io. IEEE Internet of Things Journal, 10(18), pp.16548-16563.
Chowdhury, K., Lamacchia, D., Frenk Feldman, V., Mallik, A., Rahman, I. and Alam, Z., 2020, November. A Cloud–Based Smart Engineering and Predictive Computation System for Pipeline Design and Operation Cost Reduction. In Abu Dhabi International Petroleum Exhibition and Conference (p. D012S116R200). SPE.
Colonnelli, I., Cantalupo, B., Spampinato, C., Pennisi, M. and Aldinucci, M., 2021. Bringing AI pipelines onto cloud-HPC: setting a baseline for accuracy of COVID-19 AI diagnosis. arXiv preprint arXiv:2108.01033.
García, Á.L., De Lucas, J.M., Antonacci, M., Zu Castell, W., David, M., Hardt, M., Iglesias, L.L., Moltó, G., Plociennik, M., Tran, V. and Alic, A.S., 2020. A cloud-based framework for machine learning workloads and applications. IEEE access, 8, pp.18681-18692.
Goh, P.J., Hoe, Z.Y., Low, C.Y., Koh, C.T., Mohammad, U., Lee, K. and Tan, C.F., 2021, November. Conceptual design of cloud-based data pipeline for smart factory. In Symposium on Intelligent Manufacturing and Mechatronics (pp. 29-39). Singapore: Springer Nature Singapore.
Mani, D.R., Maynard, M., Kothadia, R., Krug, K., Christianson, K.E., Heiman, D., Clauser, K.R., Birger, C., Getz, G. and Carr, S.A., 2021. PANOPLY: a cloud-based platform for automated and reproducible proteogenomic data analysis. Nature methods, 18(6), pp.580-582.
Quaranta, L., Calefato, F. and Lanubile, F., 2021. A taxonomy of tools for reproducible machine learning experiments. AIxIA 2021.
Spjuth, O., Frid, J. and Hellander, A., 2021. The machine learning life cycle and the cloud: implications for drug discovery. Expert opinion on drug discovery, 16(9), pp.1071-1079.
Xin, D., Miao, H., Parameswaran, A. and Polyzotis, N., 2021, June. Production machine learning pipelines: Empirical analysis and optimization opportunities. In Proceedings of the 2021 International Conference on Management of Data (pp. 2639-2652).
Xin, D., Wu, E.Y., Lee, D.J.L., Salehi, N. and Parameswaran, A., 2021, May. Whither automl? understanding the role of automation in machine learning workflows. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-16).
Chenchala, P. K., Choppadandi, A., Kaur, J., Nakra, V., & Pandian, P. K. G. (2020). Predictive Maintenance and Resource Optimization in Inventory Identification Tool Using ML. International Journal of Open Publication and Exploration, 8(2), 43-50. https://ijope.com/index.php/home/article/view/127
Predictive Maintenance and Resource Optimization in Inventory Identification Tool Using ML. International Journal of Open Publication and Exploration, 8(2), 43-50. https://ijope.com/index.php/home/article/view/127
Fadnavis, N. S., Patil, G. B., Padyana, U. K., Rai, H. P., & Ogeti, P. (2020). Machine learning applications in climate modeling and weather forecasting. NeuroQuantology, 18(6), 135-145. https://doi.org/10.48047/nq.2020.18.6.NQ20194
Tilala, Mitul, and Abhip Dilip Chawda. "Evaluation of Compliance Requirements for Annual Reports in Pharmaceutical Industries." NeuroQuantology 18, no. 11 (November 2020): 138-145. https://doi.org/10.48047/nq.2020.18.11.NQ20244.
AI-Driven Customer Relationship Management in PK Salon Management System. (2019). International Journal of Open Publication and Exploration, ISSN: 3006-2853, 7(2), 28-35. https://ijope.com/index.php/home/article/view/128
Mitul Tilala, Abhip Dilip Chawda, Abhishek Pandurang Benke, Akshay Agarwal. (2022). Regulatory Intelligence: Leveraging Data Analytics for Regulatory Decision-Making. International Journal of Multidisciplinary Innovation and Research Methodology, ISSN: 2960-2068, 1(1), 78–83. Retrieved from https://ijmirm.com/index.php/ijmirm/article/view/77
Tilala, Mitul, and Abhip Dilip Chawda. "Evaluation of Compliance Requirements for Annual Reports in Pharmaceutical Industries." NeuroQuantology 18, no. 11 (November 2020): 138-145. https://doi.org/10.48047/nq.2020.18.11.NQ20244.
Kamuni, Navin, Suresh Dodda, Venkata Sai Mahesh Vuppalapati, Jyothi Swaroop Arlagadda, and Preetham Vemasani. "Advancements in Reinforcement Learning Techniques for Robotics." Journal of Basic Science and Engineering 19, no. 1 (2022): 101-111. ISSN: 1005-0930.
Narukulla, Narendra, Joel Lopes, Venudhar Rao Hajari, Nitin Prasad, and Hemanth Swamy. "Real-Time Data Processing and Predictive Analytics Using Cloud-Based Machine Learning." Tuijin Jishu/Journal of Propulsion Technology 42, no. 4 (2021): 91-102.
Nitin Prasad. (2022). Security Challenges and Solutions in Cloud-Based Artificial Intelligence and Machine Learning Systems. International Journal on Recent and Innovation Trends in Computing and Communication, 10(12), 286–292. Retrieved from https://www.ijritcc.org/index.php/ijritcc/article/view/10750
Big Data Analytics using Machine Learning Techniques on Cloud Platforms. (2019). International Journal of Business Management and Visuals, ISSN: 3006-2705, 2(2), 54-58. https://ijbmv.com/index.php/home/article/view/76
Shah, J., Prasad, N., Narukulla, N., Hajari, V. R., & Paripati, L. (2019). Big Data Analytics using Machine Learning Techniques on Cloud Platforms. International Journal of Business Management and Visuals, 2(2), 54-58. https://ijbmv.com/index.php/home/article/view/76
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
