Robust Machine Learning Models for Security-Critical Applications

Abstract

A modest and human-imperceptible input perturbation may easily modify the model output entirely, as revealed by recent research. Machine learning models are susceptible to adversarial perturbations, which are also known as adversarial perturbations. Formal verification of the resilience of machine learning models is becoming more relevant as a result of the substantial security vulnerabilities that this has produced for a large number of real-world applications. This thesis investigates the resilience of tree-based models and deep neural networks, and it also takes into consideration the applications of robust machine learning models in the field of deep reinforcement learning. In the beginning, we come up with an innovative method to learn robust trees. Our technique seeks to improve performance under the worst-case perturbation of input characteristics, which gives rise to a max-min saddle point issue when splitting nodes in trees. Our method's goal is to optimize performance under these conditions. Through the process of approximating the inner minimizer in this saddle point issue, we suggest fast tree construction methods. Furthermore, we show efficient implementations for traditional information gain based trees as well as state-of-the-art tree boosting models such as XGBoost. The resilience of the model is greatly improved by our strategy, as shown by the experiments. In addition to this, we present an effective way for determining whether or not tree ensembles are resilient. The topic of verifying tree ensembles is recast as a max-clique problem on a multipartite graph by our team. We design an effective multi-level verification approach that is capable of providing tight lower limits on the resilience of decision tree ensembles. Additionally, our algorithm allows for iterative improvement and termination at any moment with no restrictions. When applied to random forest or gradient boosted decision trees models that have been trained on a variety of datasets, our algorithm is up to hundreds of times faster than the previous approach, which requires the solution of a mixed integer linear programming problem. Furthermore, our algorithm is able to provide tight robustness verification bounds on large ensembles that contain hundreds of deep trees. We submit a variety of empirical studies on the feasibility and the difficulty of adversarial training for neural networks. These findings are based on our own research. We demonstrate that even with adversarial defense, the resilience of a model on a test example has a substantial association with the distance between that example and the myriad of training data incorporated by the network. This is the case even when the adversarial defense is included. It is more probable that adversarial assaults will be successful against test samples that are quite far away from this manifold. As a consequence of this, we show that an adversarial training-based defense is susceptible to a new category of attacks known as the "blind-spot attack." This type of attack occurs when the input examples are located in low density regions (also known as "blind spots") of the empirical distribution of training data, but they are still on the valid ground-truth data manifold. In conclusion, we take neural network resilient training approaches and apply them to deep reinforcement learning (DRL) in order to train agents that are resistant to perturbations on state observations. In order to investigate the underlying characteristics of this issue, we offer the state-adversarial Markov decision process (SA-MDP). Additionally, we provide a theoretically principled regularization that can be used for a variety of deep learning and reinforcement learning algorithms, such as deep Q networks (DQN) and proximal policy optimization (PPO). We provide major improvements to the resilience of agents when they are subjected to powerful adversarial assaults via white box, including novel attacks that we have developed ourselves.