Machine Learning Approach for Regression Testing: A Case Study in Markov Chain Model
Keywords:
digitization, scalability, Regression Testing, consumption, bugAbstract
In the wake of digitization of business processes, Software has turned into an approach to carrying out the businesses while considering software testing as an investment not as an expense. Now-adays, businesses focus on prioritizing scalability so that they may allow testers to make con-tinuous integration and testing without affecting end user’s experience towards the application. Regression Testing can be performed multiple times (if required) to either to whole application under test or to only part of the application to ensure the validity of the changes made. Because of the repetitive nature of Regression Testing, software testers need to automate the process to minimize the consumption of time and effort. This paper mainly focuses on one of the machine learning techniques i.e., Markov Chain Model that randomly selecting a path to generate test cases that may further identify those test cases which are most prospective to uncover the new bugs. This can be done by observing the transition from a state in which no bugs have been found to a state in which a bug has been found.
Downloads
References
Haghighatkhah, A., Mäntylä, M., Oivo, M., & Kuvaja, P. (2018). Test Prioritization in Continuous Integration, Environments. J. Syst. Softw., 146, 80-98, https://doi.org/ 10.1016/ j.jss.2018.08.061.
Noorian, M. Bagheri, E., & Du, W. (2011). Machine Learning-Based Software Testing: Towards a Classification Framework. In SEKE, pp. 225–229.
Lou, Y., Chen, J., Zhang, L., & Hao, D. (2019). A Survey on Regression Test-Case Prioritization. In Advances in Computers; Elsevier, pp 1–46, https://doi.org/ 10.1016/zbs.adcom.2018.10.001
Hajri, I., Goknil, A., Pastore, F., & Briand, L. C. (2020). Automating System Test Case Classification and Prioritization for Use Case-Driven Testing in Product Lines, Empir. Softw. Eng., 25(5), 3711–3769. https://doi.org/ 10.1007/s10664-020-09853-4.
Kim, J., Ryu, J. W., Shin, H. J., & Song, J. H. (2017).
Machine Learning Frameworks for Automated Software Testing Tools: A Study. International Journal of Contents, 13(1), 38–44, https://doi.org/10.5392/IJoC.2017.13.1.038
Nguyen, V & Le, B. RLTCP: A Reinforcement Learning Approach to Prioritizing Automated User Interface Tests. Inf. Softw. Technol. 2021, 136 (106574), 106574. https://doi.org/10.1016/j.infsof.2021.106574.
Durelli, V. H. S., Durelli, R. S., Borges, S. S., Endo, A. T., Eler, M. M., Dias, D. R. C. & Guimaraes, M. P. (2019) Machine Learning Applied to Software Testing: A Systematic Mapping Study. IEEE Trans. Reliab., 68(3), 1189–1212. https://doi.org/10.1109/tr.2019.2892517.
Lousada, J. & Ribeiro, M. (2020) Reinforcement Learning for Test Case Prioritization. arXiv [cs.SE],. http://arxiv.org/abs/2012.11364.
Li, Q. & Pham, H. Modeling Software Fault-Detection and Fault-Correction Processes by Considering the Dependencies between Fault Amounts. Appl. Sci. (Basel) 2021, 11(15), 6998. https://doi.org/10.3390/app11156998.
Bagherzadeh, M., Kahani, N. & Briand, L. Reinforcement Learning for Test Case Prioritization. IEEE Trans. Softw. Eng. 2022, 48(8), 2836–2856. https://doi.org/10.1109/tse. 2021.3070549.
Agarwal, D. Tamir, D. E. Last, M. Kandel, A. (2012).
A Comparative Study of Artificial Neural Networks and Info-Fuzzy Networks as Automated Oracles in Software Testing, IEEE Trans. Syst. Man Cybern. A Syst. Hum.,
(5), 1183–1193. https://doi.org/10.1109/tsmca.2012. 2183590.
Path, O. C. (2019). Heuristic based Regression Test Case Prioritization Algorithm with Analysis for Test Cost Reduction and Optimized Cost Path Generation. Journal of Theoretical and Applied Information Technology, No. 17.
Lachmann, R., Schulze, S., Nieke, M., Seidl, C., &
Schaefer, I. (2016). System-Level Test Case Prioritization Using Machine Learning. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE., https://doi.org/10.1109/ICMLA.2016. 0065
Zhang, W., Qi, Y., Zhang, X., Wei, B., Zhang, M., &
Dou, Z. On Test Case Prioritization Using Ant Colony Optimization Algorithm. In 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS); IEEE, 2019. https://doi.org/10.1109/HPCC/SmartCity/DSS.2019.00388
Zhao, X., Wang, Z., Fan, X., & Wang, Z. (2015).
A Clustering-Bayesian Network Based Approach for Test Case Prioritization. In 2015 IEEE 39th Annual Computer Software and Applications Conference; IEEE,. https:// doi.org/10.1109/COMPSAC.2015.154
Mirarab, S., & Tahvildari, L. (2007). A Prioritization Approach for Software Test Cases Based on Bayesian Networks. In Fundamental Approaches to Software Engineering; Springer Berlin Heidelberg: Berlin, Heidelberg, pp. 276-290. https://doi.org/10.1007/978-3-540-71289-3_22
Stallbaum, H., Metzger, A. & Pohl, K. (2008) An Automated Technique for Risk-Based Test Case Generation and Prioritization. In Proceedings of the 3rd international workshop on Automation of software test; ACM: New York, NY, USA, https://doi.org/10.1145/1370042.1370057
Lousada, J., & Ribeiro, M. (2020). Neural Network Embeddings for Test Case Prioritization. arXiv [cs.SE], http://arxiv.org/abs/2012.10154.
Meçe, E. K., Paci, H. & Binjaku, K. (2020). The Application of Machine Learning in Test Case Prioritization-a Review. European Journal of Electrical Engineering and Computer Science, 4(1), https://doi.org/10.24018/ejece.2020. 4.1.128
Carlson, R., Do, H., & Denton, A. (2011). A Clustering Approach to Improving Test Case Prioritization: An Industrial Case Study. In 2011 27th IEEE International Conference on Software Maintenance (ICSM); IEEE., https://doi.org/10.1109/ICSM.2011.6080805
Gupta, N., Sharma, A. & Pachariya, M. K. (2022).
Multi-Objective Test Suite Optimization for Detection and Localization of Software Faults. J. King Saud Univ. - Comput. Inf. Sci., 34(6), 2897–2909. https://doi.org/ 10.1016/j.jksuci. 2020.01.009.
Cheruiyot, V. (2021). Machine Learning Driven Software Test Case Selection. University of Alberta Libraries. https://doi.org/10.7939/R3-M9D9-9861.
Xiao, H., Cao, M., & Peng, R. (2020). Artificial Neural Network-Based Software Fault Detection and Correction Prediction Models Considering Testing Effort. Applied Soft Computing, 94, https://doi.org/10.1016/j.asoc.2020.106491
Höstklint, N., & Larsson, J. (2021). Dynamic Test Case Selection Using Machine Learning.
Lima, J. A. P.; Vergilio, S. R. (2022). A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments. IEEE Trans. Softw. Eng., 48 (2), 453–465. https://doi.org/10.1109/tse.2020.2992428.
Lachmann, R. (2017). Black-Box Test Case Selection and Prioritization for Software Variants and Versions, Universitätsbibliothek Braunschweig, https://doi.org/ 10.24355/dbbs.084-201711021237
Busjaeger, B. & Xie, T. (2016). Learning for Test Prioritization: An Industrial Case Study. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering; ACM: New York, NY, USA, https://doi.org/10.1145/2950290.2983954
Jahan, H., Feng, Z., Mahmud, S. M. H. & Dong, P. (2019). Version Specific Test Case Prioritization Approach Based on Artificial Neural Network. J. Intell. Fuzzy Syst. 2019,
(6), 6181–6194, https://doi.org/10.3233/jifs-181998.
Chen, J., Bai, Y., Hao, D., Xiong, Y., Zhang, H., Zhang, L., & Xie, B. Test Case Prioritization for Compilers: A Text-Vector Based Approach. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST); IEEE, 2016. https://doi.org/10.1109/ ICST.2016.19
Kashyap, A. (2013). A Markov Chain and Likelihood-Based Model Approach for Automated Test Case Generation, Validation and Prioritization: Theory and Application (Doctoral Dissertation).
Chen, J., Hu, W., Hao, D., Xiong, Y., Zhang, H., Zhang, L., & Xie, B. (2016). An Empirical Comparison of Compiler Testing Techniques. In Proceedings of the 38th International Conference on Software Engineering; ACM: New York, NY, USA, https://doi.org/10.1145/2884781.2884878
Kalyani, R., Mounika, P.S., Naveen, R., Maridu, G., &
Ramya, P. (2018). Test Case Prioritization Using Requirements Clustering. International Journal of Applied Engineering Research, 13(15), 11776–11780.
Chen, J., Bai, Y., Hao, D., Xiong, Y., Zhang, H. & Xie, B. (2017). Learning to Prioritize Test Programs for Compiler Testing. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE); IEEE, https://doi.org/ 10.1109/ICSE.2017.70
Spieker, H. Gotlieb, A. Marijan, D & Mossige, M. (2017). Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration.
In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis; ACM: New York, NY, USA, https://doi.org/10.1145/3092703.3092709
Tonella, P., Avesani, P., & Susi, A. Using the Case-Based Ranking Methodology for Test Case Prioritization. In 2006 22nd IEEE International Conference on Software Maintenance; IEEE, 2006. https://doi.org/10.1109/ICSM. 2006.74
Konsaard, P., & Ramingwong, L. (2015). Total Coverage Based Regression Test Case Prioritization Using Genetic Algorithm. In 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON); IEEE, https://doi.org/10.1109/ECTICon.2015.7207103
Improving Proceeding Test Case Prioritization with Learning Software Agents. (2014). In Proceedings of the 6th International Conference on Agents and Artificial Intelligence; SCITEPRESS - Science and and Technology Publications, https://doi.org/10.5220/0004920002930298
Lachmann, R. (2018). 12.4 - Machine Learning-Driven Test Case Prioritization Approaches for Black-Box Software Testing. In Proceeding - ettc2018; AMA Service GmbH, Von-Münchhausen-Str. 49, 31515 Wunstorf, Germany, https://doi.org/10.5162/ettc2018/12.4
Sharif, A., Marijan, D., & Liaaen, M. (2021). DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME);
IEEE, https://doi.org/10.1109/ICSME52107.2021.00053
Matcha, W., Touré, F., Badri, M., & Badri, L. (2020).
Using Deep Learning Classifiers to Identify Candidate Classes for Unit Testing in Object-Oriented Systems. In SEKE,
pp 353–358.
Medhat, N., Moussa, S. M., Badr, N. L., Tolba, M. F. A. (2020). Framework for Continuous Regression and Integration Testing in IoT Systems Based on Deep
Learning and Search-Based Techniques. IEEE Access, 8, 215716–215726. https://doi.org/10.1109/access.2020.3039 931.
Landin, C., Hatvani, L., Tahvili, S., Haggren, H., Längkvist, M., Loutfi, A., & Håkansson, A. (2020). Performance Comparison of Two Deep Learning Algorithms in Detecting Similarities between Manual Integration Test Cases. In The Fifteenth International Conference on Software Engineering Advances (ICSEA 2020); Porto, Portugal, pp. 90–97.
Chen, Y., Wang, Z., Wang, D., Yao, Y., & Chen, Z. (2019). Behavior Pattern-Driven Test Case Selection for Deep Neural Networks. In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest); IEEE, https://doi. org/10.1109/AITest.2019.000-2
Omri, S., & Sinz, C. (2021). Machine Learning Techniques for Software Quality Assurance: A Survey. arXiv [cs.SE],. http://arxiv.org/abs/2104.14056.
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (1999). Test Case Prioritization: An Empirical Study. In Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM’99). “Software Maintenance for Business Change” (Cat. No.99CB36360); IEEE, https://doi.org /10.1109/ICSM.1999.792604
Gökçe, N., Eminov, M., & Belli, F. Coverage-Based, Prioritized Testing Using Neural Network Clustering.
In Computer and Information Sciences – ISCIS 2006; Springer Berlin Heidelberg: Berlin, Heidelberg, 2006,
pp. 1060–1071. https://doi.org/10.1007/11902140_110
Gökçe, N., Belli̇, F., Emi̇nli̇, M., & Di̇nçer, B. T. (2015). Model-Based Test Case Prioritization Using Cluster Analysis: A Soft-Computing Approach. TURK. J. OF ELECTR. ENG. COMPUT. SCI. 23, 623–640. https://doi.org/10.3906/elk-1209-109.
Koochakzadeh, N. & Garousi, V. A (2010). Tester-Assisted Methodology for Test Redundancy Detection. Adv. Softw. Eng., pp. 1–13. https://doi.org/10.1155/2010/ 932686.
Singh, Y., Kaur, A., & Suri, B. (2010). Test Case Prioritization Using Ant Colony Optimization. Softw. Eng. Notes, 35(4), 1–7. https://doi.org/10.1145/1811226. 1811238.
Paramshetti, P., & Phalke, D. A. (2014). Survey on Software Defect Prediction Using Machine Learning Techniques. International Journal of Science and Research, 3(12),
–1397.
Gokce, N. & Eminli, M. (2014). Model-Based Test Case Prioritization Using Neural Network Classification. Comput. Sci. Eng. Int. J., 4 (1), 15–25. https://doi.org/ 10.5121/cseij.2014.4102.
Ribeiro, M., Grolinger, K., Capretz, M. A. M. (2015).
MLaaS: Machine Learning as a Service. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA); IEEE, https://doi.org/10.1109/ ICMLA.2015.152
Jiang, B., & Chan, W. K. (2015). Input-Based Adaptive Randomized Test Case Prioritization: A Local Beam Search Approach. J. Syst. Softw., 105, 91–106. https://doi.org/ 10.1016/ j.jss.2015.03.066.
Yan, R., Chen, Y., Gao, H., & Yan, J. (2022). Test Case Prioritization with Neuron Valuation-Based Pattern. Science of Computer Programming, 215. https://doi.org/10.1016/ j.scico.2021.102761
Korel, B., & Koutsogiannakis, G. (2009). Experimental Comparison of Code-Based and Model-Based Test Prioritization. In 2009 International Conference on Software Testing, Verification, and Validation Workshops; IEEE, https://doi.org/10.1109/ICSTW.2009.45
Mirarab, S. & Tahvildari, L. (2008). An Empirical Study on Bayesian Network-Based Approach for Test Case Prioritization. In 2008 International Conference on Software Testing, Verification, and Validation; IEEE,. https:// doi.org/10.1109/ICST.2008.57
Wei, D., Sun, Q., Wang, X., Zhang, T., & Chen, B. (2020).
A Model-Based Test Case Prioritization Approach Based on Fault Urgency and Severity. Int. J. Softw. Eng. Knowl. Eng. 30(02), 263–290. https://doi.org/10.1142/s021819402050 0126.
Chen, J., Zhu, L., Chen, T. Y., Huang, R., Towey, D.,
Kuo, F.-C., & Guo, Y. An Adaptive Sequence Approach for OOS Test Case Prioritization. In 2016 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW); IEEE, 2016. https://doi.org/10 .1109/ ISSREW. 2016.29
Muthusamy, T. K. S. (2014). Effectiveness of Test Case Prioritization Techniques Based on Regression Testing.
Int. J. Softw. Eng. Appl., 5 (6), 113–123. https://doi.org/ 10.5121/ijsea.2014.5608.
King, T. M., Arbon, J., Santiago, D., Adamo, D., Chin, W., & Shanmugam, R. (2019). AI for Testing Today and Tomorrow: Industry Perspectives. In 2019 IEEE International Conference on Artificial Intelligence Testing (AITest); IEEE,. https://doi.org/10.1109/AITest.2019.000-3
Rajora , K., & abdulhussein , N. salih. (2023). Reviews research on applying machine learning techniques to reduce false positives for network intrusion detection systems. Babylonian Journal of Machine Learning, 2023, 26–30. https://doi.org/10.58496/BJML/2023/005
Alsajri, A., & Steiti, A. (2023). Intrusion Detection System Based on Machine Learning Algorithms:( SVM and Genetic Algorithm). Babylonian Journal of Machine Learning, 2024, 15–29. https://doi.org/10.58496/BJML/2024/002
Alsajri, A. (2023). A Review on Machine Learning Strategies for Real-World Engineering Applications. Babylonian Journal of Machine Learning, 2023, 1–6.
https://doi.org/10.58496/BJML/2023/001
T. Bhaskar, S. A. Shiney, S. B. Rani, K. Maheswari, S. Ray and V. Mohanavel, "Usage of Ensemble Regression Technique for Product Price Prediction," 2022 4th International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2022, pp. 1439-1445, doi: 10.1109/ICIRCA54612.2022.9985521.
Saravanan, P., Aparna Pandey, Kapil Joshi, Ruchika Rondon, Jonnadula Narasimharao, and Afsha Akkalkot Imran. "Using machine learning principles, the classification method for face spoof detection in artificial neural networks." In 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), pp. 2784-2788. IEEE, 2023.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
