A Hybrid Machine Learning Model for Demand Forecasting: Combination of K-means, Elastic-Net, and Gaussian Process Regression

Authors

  • Doohee Chung Corresponding Author, School of Global Entrepreneurship and Information Communication Technology (ICT), Handong Global University - 05319, Republic of Korea
  • Chan Gyu Lee Graduate School of Data Science, Seoul National University - 05319, Republic of Korea
  • Sungmin Yang School of Global Entrepreneurship and Information Communication Technology (ICT), Handong Global University - 05319, Republic of Korea

Keywords:

Demand Forecast, Gaussian Process Regression, K-means, ElasticNet, Hybrid Machine Learning

Abstract

Accurate prediction of demand assists companies in responding flexibly to uncertain market conditions. However, companies face difficulties due to the wide variety of product types and the uncertainty of their scale. To solve this problem, this study proposes a hybrid model that combines K-means, ElasticNet, and Gaussian process regression (GPR). GPR effectively addresses the issue of nonlinear prediction, and its performance is excellent when the model is trained with clusters of similar data instead of training the entire dataset at once. Moreover, the model enhances its training process by extracting crucial and contributing variables for each cluster. To implement these techniques, this study utilizes K-means and ElasticNet in combination with GPR. The model is applied to a case study of a U.S. manufacturing company and is compared with other benchmarking models, including single GPR, K-means+GPR, and ElasticNet+GPR, to evaluate its performance. The hybrid model achieved the best prediction accuracy, recording a mean absolute error (MAE) of 5.57, demonstrating its potential as foundational research for constructing an efficient demand forecasting model under uncertain conditions.

Downloads

Download data is not yet available.

References

F. Amini and G. Hu, “A two-layer feature selection method using genetic algorithm and elastic net,” Expert Systems with Applications, vol. 166, 2021, 114072.

Y. Aviv, “On the benefits of collaborative forecasting partnerships between retailers and manufacturers,” Management Science, vol. 53, no. 5, 2007, pp. 777-794.

P. K. Bala, “Improving inventory performance with clustering based demand forecasts,” Journal of Modelling in Management, vol. 7, no. 1, 2012, pp. 23-37.

K. Bandara, P. Shi, C. Bergmeir, H. Hewamalage, Q. Tran and B. Seaman, “Sales demand forecast in e-commerce using a long short-term memory neural network methodology,” in International conference on neural information processing, vol. 11955, 2019, pp. 462-474.

Y. Bengio, O. Delalleau and N. Roux, “The curse of highly variable functions for local kernel machines,” Advances in neural information processing systems, vol. 18, 2005.

J. Bibaud-Alves, P. Thomas and H. Bril, “Demand forecasting using artificial neuronal networks and time series: application to a French furniture manufacturer case study,” in 11th International Joint Conference on Computational Intelligence, IJCCI’19 (11th International Conference on Neural Computation Theory and Applications, NCTA), vol. 1, 2019.

V. Bolón-Canedo, N. Sánchez-Maroño and A. Alonso-Betanzos, “A review of feature selection methods on synthetic data,” Knowledge and information systems, vol. 34, no. 3, 2013, pp. 483-519.

N. Burkart and M. F. Huber, “A survey on the explainability of supervised machine learning,” Journal of Artificial Intelligence Research, vol. 70, 2021, pp. 245-317.

I. F. Chen and C. J. Lu, “Demand Forecasting for Multichannel Fashion Retailers by Integrating Clustering and Machine Learning Algorithms,” Processes, vol. 9, no. 9, 2021, 1578.

E. Choi, S. Cho, and D. K. Kim, “Power demand forecasting using long short-term memory (LSTM) deep-learning model for monitoring energy sustainability,” Sustainability, 12(3) (2020) 1109.

M. Christopher, and H. Lee, “Mitigating supply chain risk through improved confidence,” International journal of physical distribution & logistics management, vol. 34, no. 5, 2004.

D. Dzyabura, and S. Jagabathula, “Offline assortment optimization in the presence of an online channel,” Management Science, vol. 64, no. 6, 2018, pp. 2767-2786.

S. J. Gershman, and D. M. Blei, “A tutorial on Bayesian nonparametric models,” Journal of Mathematical Psychology, 56(1) (2012) 1-12.

L. Guo, W. Fang, Q. Zhao, and X. Wang, “The hybrid PROPHET-SVR approach for forecasting product time series demand with seasonality,” Computers & Industrial Engineering, vol. 161, 2021, 107598.

K. H. Huarng, T. H. K. Yu, and T. T. Kao, “Analyzing structural changes using clustering techniques,” International Journal of Innovative Computing, Information and Control, vol. 4, no. 5, 2008, pp. 1195-1202.

S. Ji, X. Wang, W. Zhao, and D. Guo, “An application of a three-stage XGBoost-based model to sales forecasting of a cross-border e-commerce enterprise,” Mathematical Problems in Engineering, vol. 2019, 2019.

M. A. Khan, S. Saqib, T. Alyas, A. U. Rehman, Y. Saeed, A. Zeb, M. Zareei and E. M. Mohamed, “Effective demand forecasting model using business intelligence empowered with machine learning,” IEEE Access, vol. 8, 2020, pp. 116013-116023.

K. Krauth, E. V. Bonilla, K. Cutajar and M. Filippone, “AutoGP: Exploring the capabilities and limitations of Gaussian process models,” arXiv preprint arXiv:1610.05392, vol. 1, 2016.

B. Lakshmanan, P. S. N. Vivek Raja and V. Kalathiappan, “Sales demand forecasting using LSTM network,” in Artificial Intelligence and Evolutionary Computations in Engineering Systems, vol. 1056, 2020, pp. 125-132.

R. Law, G. Li, D. K. C. Fong, and X. Han, “Tourism demand forecasting: A deep learning approach,” Annals of tourism research, vol. 75, 2019, pp. 410-423.

J. MacQueen, “Classification and analysis of multivariate observations,” in 5th Berkeley Symp. Math. Statist. Probability, vol. 1, 1967, pp. 281-297.

M. Nagashima, F. T. Wehrle, L. Kerbache and M. Lassagne, “Impacts of adaptive collaboration on demand forecasting accuracy of different product categories throughout the product life cycle,” Supply chain management: an international journal, vol. 20, no. 4, 2015.

J. Noh, H. J. Park, J. S. Kim and S. J. Hwang, “Gated recurrent unit with genetic algorithm for product demand forecasting in supply chain management,” Mathematics, vol. 8, no. 4, 2020, 565.

S. Punia, K. Nikolopoulos, S. P. Singh, J. K. Madaan and K. Litsiou, “Deep learning with long short-term memory networks and random forests for demand forecasting in multi-channel retail,” International journal of production research, vol. 58, no. 16, 2020, pp. 4964-4979.

J. Quinonero-Candela and C. E. Rasmussen, “A unifying view of sparse approximate Gaussian process regression,” The Journal of Machine Learning Research, vol. 6, 2005, pp. 1939-1959.

J. Quinonero-Candela, C. E. Rasmussen and C. K. Williams, “Approximation methods for Gaussian process regression,” in Large-scale kernel machines, MIT Press, Cambridge, MA, USA, 2007, pp. 203-223.

T. Salloom, O. Kaynak and W. He, “A novel deep neural network architecture for real-time water demand forecasting,” Journal of Hydrology, vol. 599, 2021, 126353.

E. Schulz, M. Speekenbrink and A. Krause, “A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions,” Journal of Mathematical Psychology, vol. 85, 2018, pp. 1-16.

M. Seyedan, and F. Mafakheri, “Predictive big data analytics for supply chain demand forecasting: methods, applications, and research opportunities,” Journal of Big Data, vol. 7, no. 1, 2020, pp. 1-22.

T. Tanizaki, T. Hoshino, T. Shimmura and T. Takenaka, “Demand forecasting in restaurants using machine learning and statistical analysis,” Procedia CIRP, vol. 79, 2019, pp. 679-683.

S. Thomassey, “Sales forecasts in clothing industry: The key success factor of the supply chain management,” International Journal of Production Economics, vol. 128, no. 2, 2010, pp. 470-483.

R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, 1996, pp. 267-288.

C. E. Rasmussen and C. K. Williams, “Gaussian processes for machine learning,” MIT Press, Cambridge, MA, USA, 2006, pp. 4.

L. Rokach, and O. Maimon, “Clustering methods,” in Data mining and knowledge discovery handbook, Springer, Berlin, 2005, pp. 321-352.

M. Rosienkiewicz, E. Chlebus and J. Detyna, “A hybrid spares demand forecasting method dedicated to mining industry,” Applied Mathematical Modelling, vol. 49, 2017, pp. 87-107.

J. Van Belle, T. Guns and W. Verbeke, “Using shared sell-through data to forecast wholesaler demand in multi-echelon supply chains,” European Journal of Operational Research, vol. 288, no. 2, 2021, pp. 466-479.

R. M. Van Steenbergen and M. R. Mes, “Forecasting demand profiles of new products,” Decision support systems, 139 (2020) 113401.

F. Wang, M. Li, Y. Mei and W. Li, “Time series data mining: A case study with big data analytics approach,” IEEE Access, vol. 8, 2020, pp. 14322-14328.

Y. Wang, B. Feng, Q. S. Hua and L. Sun, “Short-term solar power forecasting: A combined long short-term memory and gaussian process regression method,” Sustainability, vol. 13, no. 7, 2021, 3665.

C. K. Williams, “Prediction with Gaussian processes: From linear regression to linear prediction and beyond,” in Learning in graphical models, vol. 89, 1998, pp. 599-621.

M. Yedla, S. R. Pathakota and T. M. Srinivasa, “Enhancing K-means clustering algorithm with improved initial center,” International Journal of computer science and information technologies, vol. 1, no. 2, 2010, pp. 121-125.

Y. Zhang, G. Li, B. Muskat and R. Law, “Tourism demand forecasting: A decomposed deep learning approach,” Journal of Travel Research, vol. 60, no. 5, 2021, pp. 981-997.

Z. Zhang, H. Qin, Y. Liu, Y. Wang, L. Yao, Q. Li, J. Li and S. Pei, “Long short-term memory network based on neighborhood gates for processing complex causality in wind speed prediction,” Energy Conversion and Management, vol. 192, 2019, pp. 37-51.

Z. Zhang, C. Wang, X. Peng, H. Qin, H. Lv, J. Fu and H. Wang, “Solar Radiation Intensity Probabilistic Forecasting Based on K-Means Time Series Clustering and Gaussian Process Regression,” IEEE Access, vol. 9, 2021, pp. 89079-89092

H. Zou and T. Hastie, “Regularization and variable selection via the elastic net,” Journal of the royal statistical society: series B (statistical methodology), vol. 67, no. 2, 2005, pp. 301-320.

Model Framework

Downloads

Published

17.05.2023

How to Cite

Chung, D. ., Lee, C. G. ., & Yang, S. . (2023). A Hybrid Machine Learning Model for Demand Forecasting: Combination of K-means, Elastic-Net, and Gaussian Process Regression. International Journal of Intelligent Systems and Applications in Engineering, 11(6s), 325–336. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/2859

Issue

Section

Research Article