Assessing Neural Network Robustness via Adversarial Pivotal Tuning

by   Peter Ebert Christensen, et al.

The ability to assess the robustness of image classifiers to a diverse set of manipulations is essential to their deployment in the real world. Recently, semantic manipulations of real images have been considered for this purpose, as they may not arise using standard adversarial settings. However, such semantic manipulations are often limited to style, color or attribute changes. While expressive, these manipulations do not consider the full capacity of a pretrained generator to affect adversarial image manipulations. In this work, we aim at leveraging the full capacity of a pretrained image generator to generate highly detailed, diverse and photorealistic image manipulations. Inspired by recent GAN-based image inversion methods, we propose a method called Adversarial Pivotal Tuning (APT). APT first finds a pivot latent space input to a pretrained generator that best reconstructs an input image. It then adjusts the weights of the generator to create small, but semantic, manipulations which fool a pretrained classifier. Crucially, APT changes both the input and the weights of the pretrained generator, while preserving its expressive latent editing capability, thus allowing the use of its full capacity in creating semantic adversarial manipulations. We demonstrate that APT generates a variety of semantic image manipulations, which preserve the input image class, but which fool a variety of pretrained classifiers. We further demonstrate that classifiers trained to be robust to other robustness benchmarks, are not robust to our generated manipulations and propose an approach to improve the robustness towards our generated manipulations. Code available at:


Gradient Adjusting Networks for Domain Inversion

StyleGAN2 was demonstrated to be a powerful image generation engine that...

StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing

A significant research effort is focused on exploiting the amazing capac...

Style Transformer for Image Inversion and Editing

Existing GAN inversion methods fail to provide latent codes for reliable...

Editing Out-of-domain GAN Inversion via Differential Activations

Despite the demonstrated editing capacity in the latent space of a pretr...

Ensembling with Deep Generative Views

Recent generative models can synthesize "views" of artificial images tha...

Domain Expansion of Image Generators

Can one inject new concepts into an already trained generative model, wh...

Hierarchical Semantic Regularization of Latent Spaces in StyleGANs

Progress in GANs has enabled the generation of high-resolution photoreal...

Please sign up or login with your details

Forgot password? Click here to reset