Enhancing Predictive Accuracy in Phishing Attack Detection: A Study on the Impact of Collinearity and Feature Selection in ML-based Logistic Regression Models

Abstract

Phishing threats present dangers, for people and businesses alike emphasizing the need, for creating reliable detection techniques. It is crucial to establish phishing tactics to protect confidential data and avoid monetary damages. This study delves deeper into the intricacies of logistic regression models and how these models could effectively detect phishing attacks with a focus on impact of factors like collinearity and feature selection on predictive accuracy and model performance. In addition to logistic regression, different machine learning models, such as Decision Tree Classifier, Gaussian Naive Bayes, Logistic Regression, K Nearest Neighbors and Linear Discriminant Analysis were also considered to analyze the relationships between predictor variables and successful phishing attack likelihood and the predictive accuracy from each of the methods. By conducting experiments and comparisons we show that addressing collinearity issues and employing feature selection techniques significantly improve the predictive accuracy of logistic regression models compared to other common machine learning models. Through a methodical process of feature engineering focused on addressing collinearity among predictors, we achieved a substantial reduction of over 35% in the false negative rate for the logistic regression model which is crucial as false negatives are more costly. These findings provide insights, for enhancing the efficiency of phishing detection systems to strengthen cybersecurity defenses against emerging threats.