Enhanced Malware Detection and Prevention using Deep Reinforcement Learning
Keywords:
Deep Reinforcement Learning, Malware Detection, Virtualization Technologies, Real-Time Threat Adaptation, Cybersecurity FrameworksAbstract
Advanced malware challenges traditional cybersecurity methods, including static signature-based detection and conventional machine learning, due to their high false-positive rates and inability to detect evolving threats. This paper proposes an adaptive framework combining Deep Reinforcement Learning (DRL) and virtualization technologies for malware detection, prediction, and prevention. DRL enables real-time threat adaptation and decision-making, while virtualization tools like Docker and VMware provide isolated environments for securely analyzing suspicious processes, ensuring system stability and reducing risks. The proposed architecture addresses scalability concerns, enhances detection accuracy, and minimizes false positives, making it suitable for diverse cybersecurity scenarios. This work establishes a foundation for integrating advanced AI techniques with virtualization to develop resilient solutions for evolving threats.
Downloads
References
Umm-e-Hani Tayyab, Faiza Babar Khan, Muhammad Hanif Durad, Asifullah Khan and Yeon Soo Lee. (2022). A survey of the recent trends in deep learning based malware detection. Journal of Cybersecurity and Privacy, 2(4), 800–829. https://doi.org/10.3390/jcp2040041
Esraa Saleh Alomari, Riyadh Rahef Nuiaa, Zaid Abdi Alkareem Alyasseri, Husam Jasim Mohammed, Nor Samsiah Sani, Mohd Isrul Esa and Bashaer Abbuod Musawi. (2023). Malware detection using deep learning and correlation-based feature selection. Symmetry, 15(1), 1-21. https://doi.org/10.3390/sym15010123
Asmaa Halbouni, Teddy Surya Gunawan, Mohamed Hadi Habaebi, Murad Halbouni, Mira Kartiwi And Robiah Ahmad. (2022). Machine learning and deep learning approaches for cybersecurity: A review. IEEE Access, 10, 19572–19585. https://doi.org/10.1109/ACCESS.2022.3151248
Jeremy Watts, Franco van Wyk, Shahrbanoo Rezaei, Yiyang Wang, Neda Masoud and Anahita Khojandi. (2022). A dynamic deep reinforcement learning-Bayesian framework for anomaly detection. IEEE Transactions on Intelligent Transportation Systems, 23(12), 22884–22894. https://doi.org/10.1109/TITS.2022.3200906
https://doi.org/10.1109/TNNLS.2021.3121870
Jannatul Ferdous, Rafiqul Islam, Arash Mahboubi and Md Zahidul Islam. (2024). AI-based ransomware detection: A comprehensive review. IEEE Access, 12, 136666–136695. https://doi.org/10.1109/ACCESS.2024.3461965
Jun Zhang, Lei Pan, Qing-Long Han, Chao Chen, Sheng Wen and Yang Xiang. (2022). Deep learning-based attack detection for cyber-physical system cybersecurity: A survey. IEEE/CAA Journal of Automatica Sinica, 9(3), 377–391. https://doi.org/10.1109/JAS.2021.1004261
Robertas Damaševičius, Algimantas Venˇckauskas, Jevgenijus Toldinas and Šarunas Grigaliunas. (2021). Ensemble-based classification using neural networks and machine learning models for Windows PE malware detection. Electronics, 10(4), 485. https://doi.org/10.3390/electronics10040485
Lan Zhang , Peng Liu, Yoon-Ho Choi and Ping Chen. (2023). Semantics-preserving reinforcement learning attack against graph neural networks for malware detection. IEEE Transactions on Dependable and Secure Computing, 20(2), 1390–1402. https://doi.org/10.1109/TDSC.2022.3153844
Downloads
Published
How to Cite
Issue
Section
License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.


