Efficient Feature Engineering-Based Anomaly Detection for Network Security
Keywords:
Network security, Feature engineering, Anomaly detection, Dimensionality reduction, Packet captureAbstract
With the rapid advancement of internet technology, network-based attacks have become increasingly prevalent, posing significant challenges to ensuring the security of network infrastructures. In response, feature selection and feature reduction have emerged as essential techniques for dealing with the large volumes of data inherent in network security applications. However, traditional feature selection methods may not always suffice when all attributes are crucial for anomaly detection. To address this, we propose a feature engineering-based approach that combines feature selection and feature extraction to effectively reduce dimensionality while preserving relevant attributes. Specifically, we introduce a Stochastic-based Feature Engineering (S_FE) algorithm tailored for both manual packet capture and real-time payload datasets. In manual packet capture datasets, our algorithm extracts Trust Value, Byte Frequency Analysis (BFA), Byte Entropy (BE), Payload Length (PL), and Stream Index features, while for real-time payload datasets, it focuses on Trust Value, direction, and Hash Value features. We compare the performance of our S_FE algorithm against widely used Feature Engineering (FE) algorithms using key metrics such as accuracy, precision, recall, and F1-score. Experimental results demonstrate the superior performance of our proposed algorithm, highlighting its efficacy in network anomaly detection. This research contributes to the development of efficient techniques for enhancing network security in the face of evolving cyber threats.
Downloads
References
Zhang, Y.; Wang, Z. Feature Engineering and Model Optimization Based Classification Method for Network Intrusion Detection. Appl. Sci. 2023, 13, 9363. https://doi.org/10.3390/app13169363
Sapna Sadhwani, Asmi Sriwastawa, Raja Muthalagu et al. Intelligent Feature Engineering Based Intrusion Detection System for IoT Network Security, 20 February 2024, PREPRINT (Version 1) available at Research Square [https://doi.org/10.21203/rs.3.rs-3961151/v1]W.-K. Chen, Linear Networks and Systems. Belmont, CA, USA: Wadsworth, 1993, pp. 123–135.
M. Panda, A. A. A. Mousa and A. E. Hassanien, "Developing an Efficient Feature Engineering and Machine Learning Model for Detecting IoT-Botnet Cyber Attacks," in IEEE Access, vol. 9, pp. 91038-91052, 2021, doi: 10.1109/ACCESS.2021.3092054.
Ruizhe Yao, Ning Wang, Zhihui Liu, Peng Chen, Di Ma, Xianjun Sheng, Intrusion detection system in the Smart Distribution Network: A feature engineering based AE-LightGBM approach, Energy Reports,Volume 7, Supplement 7,2021,Pages 353-361,ISSN 2352-4847,https://doi.org/10.1016/j.egyr.2021.10.024.
Saif, S., Yasmin, N. & Biswas, S. Feature engineering based performance analysis of ML and DL algorithms for Botnet attack detection in IoMT. Int J Syst Assur Eng Manag 14 (Suppl 1), 512–522 (2023). https://doi.org/10.1007/s13198-023-01883-7
Xinwei Zhang, Yaoci Han, Wei Xu, Qili Wang, HOBA: A novel feature engineering methodology for credit card fraud detection with a deep learning architecture,Information Sciences, Volume 557, 2021, Pages 302-316 https://doi.org/10.1016/j.ins.2019.05.023.
et. al., A. N. , . (2021). Feature Engineering based on Hybrid Features for Malware Detection over Android Framework. Turkish Journal of Computer and Mathematics Education (TURCOMAT), 12(10), 2856–2864.
Liu Z, Wang Y, Feng F, Liu Y, Li Z, Shan Y. A DDoS Detection Method Based on Feature Engineering and Machine Learning in Software-Defined Networks. Sensors. 2023; 23(13):6176. https://doi.org/10.3390/s23136176
A. Ghubaish, Z. Yang, A. Erbad and R. Jain, "LEMDA: A Novel Feature Engineering Method for Intrusion Detection in IoT Systems," in IEEE Internet of Things Journal, vol. 11, no. 8, pp. 13247-13256, 15 April15, 2024, doi: 10.1109/JIOT.2023.3328795.
M. S. Akter, H. Shahriar, J. R. Cardenas, S. Iqbal Ahamed and A. Cuzzocrea, "Feature Engineering-Based Detection of Buffer Overflow Vulnerability in Source Code Using Neural Networks," 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), Torino, Italy, 2023, pp. 765-776, doi: 10.1109/COMPSAC57700.2023.00106.
Hazman, C., Benkirane, S., Guezzaz, A., Azrour, M., Abdedaime, M. (2023). Intrusion Detection Framework for IoT-Based Smart Environments Security. In: Farhaoui, Y., Rocha, A., Brahmia, Z., Bhushab, B. (eds) Artificial Intelligence and Smart Environment. ICAISE 2022. Lecture Notes in Networks and Systems, vol 635. Springer, Cham. https://doi.org/10.1007/978-3-031-26254-8_79
Abbasi, N., Soltanaghaei, M. & Zamani Boroujeni, F. Anomaly detection in IOT edge computing using deep learning and instance-level horizontal reduction. J Supercomput 80, 8988–9018 (2024). https://doi.org/10.1007/s11227-023-05771-6
Mahmoud Ragab, Maha Farouk S. Sabir, Outlier detection with optimal hybrid deep learning enabled intrusion detection system for ubiquitous and smart environment, Sustainable Energy Technologies and Assessments, Volume 52, Part D, 2022, 102311, ISSN 2213-1388, https://doi.org/10.1016/j.seta.2022.102311.
P. K. Reddy Shabad, A. Alrashide and O. Mohammed, "Anomaly Detection in Smart Grids using Machine Learning," IECON 2021 – 47th Annual Conference of the IEEE Industrial Electronics Society, Toronto, ON, Canada, 2021, pp. 1-8, doi: 10.1109/IECON48115.2021.9589851.
M. Ravinder and V. Kulkarni, "A Review on Cyber Security and Anomaly Detection Perspectives of Smart Grid," 2023 5th International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, 2023, pp. 692-697, doi: 10.1109/ICSSIT55814.2023.10060871.
Kumar, V., Patra, S.K. (2021). Feature Engineering for Machine Learning and Deep Learning Assisted Wireless Communication. In: Oliva, D., Houssein, E.H., Hinojosa, S. (eds) Metaheuristics in Machine Learning: Theory and Applications. Studies in Computational Intelligence, vol 967. Springer, Cham. https://doi.org/10.1007/978-3-030-70542-8_4
Dongqi Han, Zhiliang Wang, Wenqi Chen, Ying Zhong, Su Wang, Han Zhang, Jiahai Yang, Xingang Shi, and Xia Yin. 2021. DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS '21). Association for Computing Machinery, New York, NY, USA, 3197–3217. https://doi.org/10.1145/3460120.3484589
D. Upadhyay, J. Manero, M. Zaman and S. Sampalli, "Gradient Boosting Feature Selection With Machine Learning Classifiers for Intrusion Detection on Power Grids," in IEEE Transactions on Network and Service Management, vol. 18, no. 1, pp. 1104-1116, March 2021, doi: 10.1109/TNSM.2020.3032618.
Shahhosseini, M., Mashayekhi, H. & Rezvani, M. A Deep Learning Approach for Botnet Detection Using Raw Network Traffic Data. J Netw Syst Manage 30, 44 (2022). https://doi.org/10.1007/s10922-022-09655-7
B. Liu, Y. Zhao, Y. Kang, Y. Cao, P. Bai and Z. Xu, "A Feature Engineering-based Method for PCB Solder Paste Position Offset Prediction," 2023 6th International Symposium on Autonomous Systems (ISAS), Nanjing, China, 2023, pp. 1-6, doi: 10.1109/ISAS59543.2023.10164303.
Berghout T, Benbouzid M, Amirat Y. Towards Resilient and Secure Smart Grids against PMU Adversarial Attacks: A Deep Learning-Based Robust Data Engineering Approach. Electronics. 2023; 12(12):2554. https://doi.org/10.3390/electronics12122554
Liu H, Lang B. Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey. Applied Sciences. 2019; 9(20):4396. https://doi.org/10.3390/app9204396
V. T. Pham, T. V. Huu, M. T. Nguyen and H. -C. Le, "Advanced Feature Processing for IoT-Based Intrusion Detection System," 2023 RIVF International Conference on Computing and Communication Technologies (RIVF), Hanoi, Vietnam, 2023, pp. 37-42, doi: 10.1109/RIVF60135.2023.10471837.
Lean Yu, Xiaoming Zhang, Hang Yin, An extreme learning machine based virtual sample generation method with feature engineering for credit risk assessment with data scarcity, Expert Systems with Applications, Volume 202, 2022, 117363, ISSN 0957-4174,https://doi.org/10.1016/j.eswa.2022.117363.
Rajput, V., Mulay, P. and Mahajan, C.M. (2024), "Bio-inspired algorithms for feature engineering: analysis, applications and future research directions", Information Discovery and Delivery, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/IDD-11-2022-0118
Ti, YW., Hsin, YY., Dai, TS. et al. Feature generation and contribution comparison for electronic fraud detection. Sci Rep 12, 18042 (2022). https://doi.org/10.1038/s41598-022-22130-2
Althar, R.R., Samanta, D. The realist approach for evaluation of computational intelligence in software engineering. Innovations Syst Softw Eng 17, 17–27 (2021). https://doi.org/10.1007/s11334-020-00383-2
C. Feng, T. Li and D. Chana, "Multi-level Anomaly Detection in Industrial Control Systems via Package Signatures and LSTM Networks," 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), Denver, CO, USA, 2017, pp. 261-272, doi: 10.1109/DSN.2017.34.
Zhijian Qu, Hanxin Liu, Zixiao Wang, Juan Xu, Pei Zhang, Han Zeng,
A combined genetic optimization with AdaBoost ensemble model for anomaly detection in buildings electricity consumption, Energy and Buildings, Volume 248, 2021, 111193, ISSN 0378-7788, https://doi.org/10.1016/j.enbuild.2021.111193.
P. Kopyt et al., “Electric properties of graphene-based conductive layers from DC up to terahertz range,” IEEE THz Sci. Technol., to be published. DOI: 10.1109/TTHZ.2016.2544142.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
