Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs

by   Philipp Benz, et al.

Convolutional Neural Networks (CNNs) have become the de facto gold standard in computer vision applications in the past years. Recently, however, new model architectures have been proposed challenging the status quo. The Vision Transformer (ViT) relies solely on attention modules, while the MLP-Mixer architecture substitutes the self-attention modules with Multi-Layer Perceptrons (MLPs). Despite their great success, CNNs have been widely known to be vulnerable to adversarial attacks, causing serious concerns for security-sensitive applications. Thus, it is critical for the community to know whether the newly proposed ViT and MLP-Mixer are also vulnerable to adversarial attacks. To this end, we empirically evaluate their adversarial robustness under several adversarial attack setups and benchmark them against the widely used CNNs. Overall, we find that the two architectures, especially ViT, are more robust than their CNN models. Using a toy example, we also provide empirical evidence that the lower adversarial robustness of CNNs can be partially attributed to their shift-invariant property. Our frequency analysis suggests that the most robust ViT architectures tend to rely more on low-frequency features compared with CNNs. Additionally, we have an intriguing finding that MLP-Mixer is extremely vulnerable to universal adversarial perturbations.


page 1

page 5

page 7

page 9

page 10


Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?

Vision transformers (ViTs) have recently set off a new wave in neural ar...

SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain

Despite the success of convolutional neural networks (CNNs) in many comp...

Analyzing Adversarial Robustness of Vision Transformers against Spatial and Spectral Attacks

Vision Transformers have emerged as a powerful architecture that can out...

Estimating relative diffusion from 3D micro-CT images using CNNs

In the past several years, convolutional neural networks (CNNs) have pro...

Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy

Recently, convolutional neural networks (CNNs) have made significant adv...

BARReL: Bottleneck Attention for Adversarial Robustness in Vision-Based Reinforcement Learning

Robustness to adversarial perturbations has been explored in many areas ...

Neural Networks with Recurrent Generative Feedback

Neural networks are vulnerable to input perturbations such as additive n...

Please sign up or login with your details

Forgot password? Click here to reset