Evasion and the Fast Gradient Sign Method

A school bus image is classified correctly. We add a carefully chosen noise pattern. The changed image still looks like a school bus to a person, and the model now predicts ostrich.

The attack has two goals that pull in opposite directions: change the prediction and keep the input close to the original. If the change can be anything, the problem is trivial. Replace the bus with an ostrich. The useful attack finds a small change with a large effect.

ℹ

Perturbation Size

Let the original input be xx and the change be rr. The adversarial input is x′=x+rx' = x + r. One way to measure the size of the change is the L2 norm:

∥r∥2=∑iri2\|r\|_2 = \sqrt{\sum_i r_i^2}

Each feature change contributes to the total distance. Squaring makes a large change to one feature more expensive than a small change. L2 is one choice. L1 adds absolute changes, while L-infinity measures the largest change to any one feature.

Perturbation Budget Explorer

xx + r

The blue region contains every perturbation allowed by the selected threat model.

Choose the distance rule

L1

0.900

L2

0.652

L-infinity

0.550

Inside the attack budget

0.652 <= 0.80

L1: adds absolute changes. Its budget forms a diamond.

L2: measures straight-line distance. Its budget forms a circle.

L-infinity: limits the largest single change. Its budget forms a square.

Apply the same two-feature perturbation under L1, L2, and L-infinity budgets. Notice how the allowed region changes shape.

The norm is part of the threat model. It says which changes count as small. A perturbation can fit inside one norm budget and fall outside another, so "small attack" is incomplete unless the norm and radius are named.

Early attacks solved an optimization problem over many steps: find a small rr that makes the model predict a chosen label while keeping every pixel valid. This works, but it can be slow.

The Fast Gradient Sign Method, or FGSM, asks a cheaper question: which direction increases the loss fastest right now?

x′=x+ε⋅sign⁡(∇xJ(θ,x,y))x' = x + \varepsilon \cdot \operatorname{sign}(\nabla_x J(\theta, x, y))

  • JJ is the model's loss.
  • θ\theta represents the trained model parameters.
  • ∇xJ\nabla_x J is the gradient of the loss with respect to the input xx.
  • sign⁡\operatorname{sign} keeps only the direction for each feature: positive or negative.
  • ε\varepsilon controls how far each feature may move.

FGSM, One Number at a Time

x' = x + epsilon sign(gradient_x J(theta, x, y))

Toy classifier: z = 2x1 - x2 + 0.3, label y = 1.

1. Forward pass

First calculate the model's current probability and loss. The model is correct, so the attack has somewhere to push.

Forward pass

z = 2(0.4) - 0.6 + 0.3 = 0.500

p = sigmoid(z) = 0.622

J = -log(p) = 0.474

Input gradient

dJ/dz = p - y = -0.378

dJ/dx = (p-y)w = [-0.755, 0.378]

Signed step

sign(dJ/dx) = [-1, 1]

r = 0.18 x sign = [-0.18, 0.18]

Attacked input

x' = [0.4, 0.6] + [-0.18, 0.18]

x' = [0.22, 0.78]

New prediction

z' = -0.040

P(y=1 | x') = 0.490

J(x') = 0.713

Clean P(y=1)

0.622

Attacked P(y=1)

0.490

Loss increase

0.239

Step through a complete FGSM calculation on a toy 2-feature logistic regression (a stand-in for a full neural network). Change epsilon and watch the input gradient, signed perturbation, loss, and prediction update from the same numbers.

Why use the sign? FGSM is built around an L-infinity budget. Once the gradient tells us whether each feature should increase or decrease, the sign sends that feature to the useful edge of its allowed interval. The magnitude of the original gradient no longer matters; epsilon sets the step size.

Training differentiates the loss with respect to the model parameters and moves downhill. FGSM freezes those parameters, differentiates with respect to the input, and moves uphill. It does not need the best possible perturbation. It needs one that crosses the boundary.

Checkpoint

What changes during an FGSM attack?

←PreviousWhite-Box and Black-Box AttacksAdversarial AttacksNext→One-Pixel AttacksAdversarial Attacks