A Distributed Framework for High-Volume Time Series Data Imputation Using Ray and Cloud Infrastructure
Keywords:
Time Series Data Imputation, Distributed Computing, Ray, Cloud Infrastructure, Big Data Analytics, Machine Learning, LSTM, Parallel Processing, Fault Tolerance, Scalable Systems, Data Engineering, Real-Time Analytics.Abstract
The rapid growth of large-scale temporal data generated from healthcare systems, Internet of Things (IoT) devices, financial platforms, environmental monitoring systems, and smart infrastructures has increased the importance of efficient time series data management. However, missing values in high-volume time series datasets significantly affect the accuracy of predictive analytics, forecasting models, and real-time decision-making systems. Traditional centralized imputation techniques often face limitations related to scalability, computational overhead, and processing latency when dealing with massive datasets. This study proposes a distributed framework for high-volume time series data imputation using Ray and cloud infrastructure technologies. The framework integrates distributed task scheduling, parallel data processing, and machine learning-based imputation methods within a scalable cloud-enabled architecture. Multiple imputation techniques, including statistical methods and advanced machine learning models such as K-Nearest Neighbor (KNN), Random Forest, and Long Short-Term Memory (LSTM) networks, were evaluated under different missing data scenarios. The distributed framework demonstrated significant improvements in execution time, scalability, resource utilization, and fault tolerance compared to traditional centralized systems. Experimental findings further indicated that machine learning-based imputation models achieved higher predictive accuracy, with LSTM-based approaches producing the lowest error rates for sequential temporal datasets. Cloud-based deployment enabled dynamic resource allocation and efficient workload balancing during large-scale processing. The proposed framework offers an effective and scalable solution for real-time time series data imputation in modern big data environments and contributes to the advancement of distributed artificial intelligence and cloud-native analytics systems.
Downloads
References
A. Almeida, S. Brás, S. Sargento, and F. C. Pinto, “Time series big data: A survey on data stream frameworks, analysis and algorithms,” Journal of Big Data, vol. 10, no. 1, p. 83, 2023.
A. Nuthalapati, “Building scalable data lakes for Internet of Things (IoT) data management,” Educational Administration: Theory and Practice, vol. 29, no. 1, pp. 412–424, 2023.
A. Y. Sun and B. R. Scanlon, “How can big data and machine learning benefit environment and water management: A survey of methods, applications, and future directions,” Environmental Research Letters, vol. 14, no. 7, p. 073001, 2019.
P. R. Sarkar, “Data-driven quality assurance systems for food safety in large-scale distribution centers,” ASRC Procedia: Global Perspectives in Science and Scholarship, vol. 2, no. 1, pp. 151–192, 2022.
A. Al Mamun, “AI-enabled modeling and monitoring of data-rich advanced manufacturing systems,” Mississippi State University, 2023.
M. Asch, T. Moore, R. Badia, M. Beck, P. Beckman, T. Bidot, et al., “Big data and extreme-scale computing: Pathways to convergence-toward a shaping strategy for a future software and data ecosystem for scientific inquiry,” The International Journal of High Performance Computing Applications, vol. 32, no. 4, pp. 435–479, 2018.
E. Zeydan and J. Mangues-Bafalluy, “Recent advances in data engineering for networking,” IEEE Access, vol. 10, pp. 34449–34496, 2022.
H. S. Munawar, S. Qayyum, F. Ullah, and S. Sepasgozar, “Big data and its applications in smart real estate and the disaster management life cycle: A systematic analysis,” Big Data and Cognitive Computing, vol. 4, no. 2, p. 4, 2020.
J. Bzai, F. Alam, A. Dhafer, M. Bojović, S. M. Altowaijri, I. K. Niazi, and R. Mehmood, “Machine learning-enabled Internet of Things (IoT): Data, applications, and industry perspective,” Electronics, vol. 11, no. 17, p. 2676, 2022.
R. Rawat and R. Yadav, “Big data: Big data analysis, issues and challenges and technologies,” in IOP Conference Series: Materials Science and Engineering, vol. 1022, no. 1, 2021, p. 012014.
S. Wang, S. Di Tommaso, J. Faulkner, T. Friedel, A. Kennepohl, R. Strey, and D. B. Lobell, “Mapping crop types in southeast India with smartphone crowdsourcing and deep learning,” Remote Sensing, vol. 12, no. 18, p. 2957, 2020.
H. A. Mohna, T. Barua, M. Mohiuddin, and M. M. Rahman, “AI-ready data engineering pipelines: A review of medallion architecture and cloud-based integration models,” American Journal of Scholarly Research and Innovation, vol. 1, no. 1, pp. 319–350, 2022.
K. L. M. Ang, J. K. P. Seng, E. Ngharamike, and G. K. Ijemaru, “Emerging technologies for smart cities’ transportation: Geo-information, data analytics and machine learning approaches,” ISPRS International Journal of Geo-Information, vol. 11, no. 2, p. 85, 2022.
D. Liu, “Distributed storage security,” in Encyclopedia of Wireless Networks. Cham, Switzerland: Springer International Publishing, 2020, pp. 338–341.
H. Huang and S. Guo, “Data-driven service provisioning,” in Encyclopedia of Wireless Networks. Cham, Switzerland: Springer International Publishing, 2020, pp. 308–312.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
