EvoColony: A Hybrid Approach to Search-Based Mutation Test Suite Reduction Using Genetic Algorithm and Ant Colony Optimization
Keywords:
software testing, mutation testing, search-based mutation, genetic algorithms, ant colony optimization, metaheuristicsAbstract
The increasing complexity of software systems requires robust and efficient test suites to ensure software quality. In this context, mutation testing emerges as an invaluable method for evaluating a test suite’s the fault detection capability. Traditional approaches to test case generation and evaluation are often inadequate, particularly when applied to mutation testing, which aims to evaluate the quality of a test suite by introducing minor changes or mutations to the code. As software projects increase in scale, there is greater computational cost of employing exhaustive mutation testing techniques, leading to a need for more efficient approaches. Incorporating metaheuristics into the realm of mutation testing offers a synergistic advantage in optimizing test suites for better fault detection. Especially, combining test suite reduction methods with mutation testing produces a more computationally efficient approach compared to more exhaustive ones. This study presents a novel approach, called EvoColony, which combines intelligent search-based algorithms, specifically genetic algorithms and ant colony optimization, to reduce test cases and enhance the effectiveness of the test suit for mutation testing. Integrating both metaheuristic techniques, the research aims to optimize existing test suites, and to improve mutant detection with fewer test cases, thus improving the overall testing quality. The results of experiments conducted were compared with traditional methods, demonstrating the superior effectiveness and efficiency of the proposed hybrid approach. The findings show a significant advancement in test case reduction when using the hybrid algorithm with mutation testing methodologies, and thus ensure the quality of test suites.
Downloads
References
M. Kintis, “Effective methods to tackle the equivalent mutant problem when testing software with mutation,” Ph.D. dissertation, Department of Informatics, Athens University of Economics and Business, Athens, Greece, 2016.
T. T. Chekam, M. Papadakis, Y. Le Traon, and M. Harman, “An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption,” 2017 IEEE/ACM 39th International Conference on Software Engineering, ICSE 2017, Institute of Electrical and Electronics Engineers Inc., Jul. 2017, pp. 597–608. doi: 10.1109/ICSE.2017.61.
Y. T. Chen et al., “Revisiting the relationship between fault detection, test adequacy criteria, and test set size,” 2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020, Institute of Electrical and Electronics Engineers Inc., Sep. 2020, pp. 237–249. doi: 10.1145/3324884.3416667.
R. A. DeMillo, R. J. Lipton and F. G. Sayward, “Hints on test data selection: help for the practicing programmer,” Computer, vol. 11, no. 4, pp. 34-41, April 1978, doi: 10.1109/C-M.1978.218136.
Y. Jia and M. Harman, “An analysis and survey of the development of mutation testing,” IEEE Transactions on Software Engineering, vol. 37, no. 5. pp. 649–678, 2011. doi: 10.1109/TSE.2010.62.
A. B. Sánchez, P. Delgado-Pérez, I. Medina-Bulo, and S. Segura, “Mutation testing in the wild: findings from GitHub,” Empirical Software Engineering, vol. 27, no. 6, Nov. 2022, doi: 10.1007/s10664-022-10177-8.
M. Papadakis, M. Kintis, J. Zhang, Y. Jia, Y. Le Traon, and M. Harman, “Mutation testing advances: an analysis and survey,” Advances in Computers, vol. 112, pp. 275–378, Jan. 2019, doi: 10.1016/bs.adcom.2018.03.015.
A. Panichella and C. C. S. Liem, “What are we really testing in mutation testing for machine learning? a critical reflection,” Proceedings of the 43rd International Conference on Software Engineering: New Ideas and Emerging Results, pp. 66–70, May 2021, doi:10.1109/ICSE-NIER52604.2021.00022.
N. Jatana, B. Suri, and S. Rani, “Systematic literature review on search based mutation testing,” E-Informatica Software Engineering Journal, vol. 11, no. 1. Politechnika Wroclawska, pp. 59–76, Jan. 01, 2017. doi: 10.5277/e-Inf170103.
A. J. Offutt, “Mutation 2000: uniting the orthogonal,” Wong, W.E. (eds) Mutation Testing for the New Century, The Springer International Series on Advances in Database Systems, vol 24., 2001, Springer, Boston, MA. doi:10.1007/978-1-4757-5939-6_7.
R. G. Hamlet, “Testing programs with the aid of a compiler,” IEEE Transactions on Software Engineering, vol. SE-3, no. 4, pp. 279-290, July 1977, doi: 10.1109/TSE.1977.231145.
Y. Jia and M. Harman, “Higher order mutation testing,” Information Software Technology, vol. 51, no. 10, pp. 1379–1393, Oct. 2009, doi: 10.1016/j.infsof.2009.04.016.
L. Chen and L. Zhang, “Speeding up mutation testing via regression test selection: an extensive study,” 2018 IEEE 11th International Conference on Software Testing, Verification and Validation, ICST 2018, Institute of Electrical and Electronics Engineers Inc., May 2018, pp. 58–69. doi: 10.1109/ICST.2018.00016.
I. Leontiuc, “Continuous mutation testing in modern software development,” M.S. thesis, Department of Software Technology, Faculty EEMCS, Delft University of Technology, Delft, the Netherlands, 2017.
Y.-S. Ma and J. Offutt, “Description of method-level mutation operators for java,” George Mason University, 2005.
R. A. Silva, S. do R. Senger de Souza, and P. S. Lopes de Souza, “A systematic review on search based mutation testing,” Information Software Technology, vol. 81, pp. 19–35, Jan. 2017, doi: 10.1016/j.infsof.2016.01.017.
N. Jatana, B. Suri, and S. Rani, “Systematic literature review on search based mutation testing,” E-Informatica Software Engineering Journal, vol. 11, no. 1. Politechnika Wroclawska, pp. 59–76, Jan. 01, 2017. doi: 10.5277/e-Inf170103.
V. Sharma, D. Rakesh Kumar, D. Sanjay Tyagi, P. Scolar, and A. P. Dcsa, “A review of genetic algorithm and mendelian law,” International Journal of Science Engineering Research, vol. 7, 2016, [Online]. Available: http://www.ijser.org
S. Uzunbayir, “A genetic algorithm for the winner determination problem in combinatorial auctions,” 2018 3rd International Conference on Computer Science and Engineering (UBMK), Sarajevo, Bosnia and Herzegovina, 2018, pp. 127-132, doi: 10.1109/UBMK.2018.8566446.
M. Dorigo, M. Birattari and T. Stutzle, “Ant colony optimization,” IEEE Computational Intelligence Magazine, vol. 1, no. 4, pp. 28-39, Nov. 2006, doi: 10.1109/MCI.2006.329691.
S. Uzunbayir, “Reverse ant colony optimization for the winner determination problem in combinatorial auctions,” International Conference on Computer Science and Engineering, UBMK 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 19–24. doi: 10.1109/UBMK55850.2022.9919488.
H. Haga and A. Suehiro, “Automatic test case generation based on genetic algorithm and mutation analysis,” 2012 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia, 2012, pp. 119-123, doi: 10.1109/ICCSCE.2012.6487127.
R. R. Sahoo and M. Ray, “Metaheuristic techniques for test case generation: a review,” Research Anthology on Agile Software, Software Development, and Testing, vol. 2, IGI Global, 2021, pp. 1043–1058. doi: 10.4018/978-1-6684-3702-5.ch052.
D. Arora and A. S. Baghel, “A hybrid meta-heuristics technique for finding optimal path by software test case reduction,” International Journal of Hybrid Information Technology, vol. 8, no. 4, pp. 35–40, Apr. 2015, doi: 10.14257/ijhit.2015.8.4.05.
Palak and P. Gulia, “Hybrid swarm and ga based approach for software test case selection,” International Journal of Electrical and Computer Engineering, vol. 9, no. 6, pp. 4898–4903, 2019, doi: 10.11591/ijece.v9i6.pp49898-4903.
B. Suri and S. Singhal, “Implementing ant colony optimization for test case selection and prioritization,” International Journal on Computer Science and Engineering, pp. 1924-1932, Nov. 2011.
N. Jatana and B. Suri, “An improved crow search algorithm for test data generation using search-based mutation testing,” Neural Processing Letters, vol. 52, no. 1, pp. 767–784, Aug. 2020, doi: 10.1007/s11063-020-10288-7.
A. Tsagaris, P. Kyratsis, and G. Mansour, “The integration of genetic and ant colony algorithm in a hybrid approach,” International Journal of Intelligent Systems and Applications in Engineering, vol. 11, no. 2, pp. 336 –, Feb. 2023.
M. B. Bashir and A. Nadeem, “Improved Genetic Algorithm to Reduce Mutation Testing Cost,” IEEE Access, vol. 5, pp. 3657-3674, 2017, doi: 10.1109/ACCESS.2017.2678200.
F. Wedyan, A. Al-Shishani, and Y. Jararweh, “GaSubtle: a new genetic algorithm for generating subtle higher-order mutants,” Information (Switzerland), vol. 13, no. 7, Jul. 2022, doi: 10.3390/info13070327.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
