Summary

The article discusses adversarial attacks in machine learning, explaining their nature, causes, impact on various systems, and potential defenses.

Abstract

Adversarial attacks involve manipulating data samples to deceive machine learning models, exploiting the linearity inherent in these systems. These attacks can compromise a wide range of machine learning models, including deep learning architectures, by making imperceptible changes to inputs that lead to incorrect outputs. The article emphasizes the vulnerability of machine learning systems across different domains, such as finance and medical AI, and underscores the importance of developing robust defenses. Proposed defense strategies include reducing the linearity of classifiers, using algorithms with high VC dimensions like RBF-SVM, and employing adversarial training to enhance model resilience. The article also touches on the limitations of traditional regularization techniques and the challenges of using GANs to distinguish between real and adversarial samples.

Opinions

The author suggests that no machine learning algorithm is immune to adversarial attacks, highlighting the pervasive threat to various models.
It is implied that the excessive linearity of machine learning models, even deep learning architectures, is a fundamental cause of their susceptibility to adversarial examples.
The author expresses skepticism about the current state of autonomous systems, indicating that adversarial attacks undermine trust in these systems due to their lack of robustness.
There is an opinion that adversarial training, while beneficial, is not a panacea and that other approaches, such as non-differentiable models, should be explored.
The article conveys that despite the initial optimism, GANs have not proven to be an effective solution for detecting adversarial samples, suggesting the problem remains largely unsolved.
The author concludes with a cautiously optimistic view that neural networks, particularly those trained adversarially, can potentially offer superior security against adversarial attacks compared to other machine learning models.

Understanding the Adversarial attacks

We’ll divide this story into four sections:

What are adversarial examples?
Why do they happen?
How can they be used to compromise machine learning systems?
What are the defenses?

What are adversarial examples?

These are those data samples that look like normal samples but are perturbed in a certain way to fool the machine learning systems. For instance, in a given image not all the pixels are of the same importance, if you could identify the most important pixels (for the ML system to make a classification decision) and change them, your algorithm decision will also change and your sample will still appear to be normal.

The panda on the right still looks like a panda after the addition of adversarial noise, but now it will be classified into a completely different class by the original classifier

Here all the objects were perturbed in such a way that everything is predicted as an airplane

Why does the adversarial attack happen?

Before we point out the reason as to why they happen, we just want to let you know that no ML algorithm is safe from adversarial attacks be it Logistic regression, softmax regression, SVM, Decision tree, Nearest neighbors, or deep learning models. Adversarial examples happen because of the excessive linearity in the systems. You may think that deep learning can easily form non-linear decision boundaries, which is true, but almost every deep learning architecture is piece-wise linear. So when a point is close to the linear boundary, even a small amount of noise can push it to the other side of the decision boundary. From the below graph also you can see that in practice deep learning classifiers have a very linear responses.

Modern deep learning architectures are piece-wise linear

Often the classes have small inter-class distances thus using the fast sign gradient method we can easily change the class of the given object. The attack fast gradient sign method consists of adding a linear amount of in-perceivable noise to the image and causing a model to incorrectly classify it. This noise is calculated by multiplying the sign of the gradient with respect to the image we want to perturb by a small constant epsilon. As epsilon increases, the model is more likely to be fooled, but the perturbations become easier to identify as well. There are several methods to generate adversarial examples.

Adversarial machine learning - Wikipedia

Currently editing: (additions) Adversarial machine learning is a machine learning technique that attempts to exploit…

en.wikipedia.org

One thing I want to clarify again is that adversarial noise might look like random noise but it certainly isn’t. Every pixel is added with a different amount of noise depending on its importance in the final classification result.

How can they be used to compromise machine learning systems?

Andrej karpathy showed in his paper how to break the linear classifiers on the ImageNet dataset. This is not limited to only images. You can fool even the tabular data which is very prominent in the financial sectors. You can look at the ART library designed to create several types of adversarial attacks for both images and tabular data. Adversarial attacks are the reason we don’t trust autonomous systems. We need systems that can handle noise and are robust in different environments.

GitHub - Trusted-AI/adversarial-robustness-toolbox: Adversarial Robustness Toolbox (ART) - Python…

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction…

github.com

One key area where adversarial examples are very dangerous is medical AI. In one of my papers, I showcased that just 5 pixels are enough to render your model pretty useless. This shows that your models are not holistic and look at data in a completely wrong way. Even though from the results it looks really smart but behind all the glory it’s just a number-crunching technique. It lacks context and that’s why the risk of adversarial attacks is so high.

So, before we move to defenses let’s revise why adversarial attacks exist, almost all ML and AI algorithms are very linear at least locally, and just by finding the gradient to a certain class you can shift the data towards it and make a successful adversarial attack.

Role of Choosing Correct Loss Function

Loss functions play a very important role in the training of modern Deep learning architecture, choosing the right loss…

medium.com

What are the defenses?

As we established earlier that adversarial examples exist because of the linearity in the system, what if we can reduce this linearity somehow. Before we do that let’s talk about VC dimensions. If you can find a set of n points, so that it can be shattered by the classifier (i.e. classify all possible 2n2n labeling correctly) and you cannot find any set of n+1 points that can be shattered (i.e. for any set of n+1 points there is at least one labeling order so that the classifier can not separate all points correctly), then the VC dimension is n. VC dimension in simpler terms basically defines the linearity of the classifier.

Vapnik-Chervonenkis dimension - Wikipedia

In Vapnik-Chervonenkis theory, the Vapnik-Chervonenkis (VC) dimension is a measure of the capacity (complexity…

en.wikipedia.org

Now, do you know an algorithm that has a very high VC dimension, meaning that it can separate points in very high dimensions? The only algorithm that I know of is the Gaussian or RBF kernel of SVM, it has a VC dimension of infinite, meaning theoretically it can separate points even in an infinite dimension which no neural network can do. And exactly because of this reason RBF-SVM is robust against adversarial examples. For any other classifier other than RBF-SVM researchers can generate make any digit being predicted as 0, 1, 2…., 9 and that too without any significant noise in the input. The below image shows that when they tried to generate adversarial examples for RBF-SVM, digits actually changed a lot. RBF-SVM pushes the data in infinite dimensions and thus the inter-class distance between each class is huge.

For neural networks you can train the network with adversarial examples this tends to increase the robustness of the networks. Adversarial training provides regularization and semi-supervised learning The below image shows the performance increase with training with adversarial examples. Other ideas are to train models such that they are non-differentiable. Also, adversarial attacks don’t go away because of traditional regularization techniques.

Initially, researchers thought that GAN could solve the problem, they thought if we know the real distribution of the input we could avoid this problem turns out it is still extremely tough to identify adversarial samples. Look at the below image one set is good and the other is bad, but to human eyes, both look the same.

The left one is the adversarial example and the right one is the real example

Linear models: SVM / linear regression cannot learn a step function, so adversarial training is less useful, very similar to weight decay
k-NN: adversarial training is prone to overfitting.

CONCLUSION

Neural nets can actually become more secure than other models. Adversarially trained neural nets to have the best empirical success rate on adversarial examples of any machine learning model.
Attacking is easy
Defending is difficult
Adversarial training provides regularization and semi-supervised learning