Noise, Score, Velocity

By the end of this session you will be able to:

• Write the forward noising equation from memory and reproduce a Diffusers add_noise call with two lines of tensor math
• Convert between the three training targets, noise, clean sample and velocity, in either direction, and say which one a checkpoint expects
• Say what the score function points at, and recover a clean sample estimate from it with Tweedie's formula

Noising is one equation, not a thousand steps

"Forward diffusion process" makes people picture a loop that adds a little noise a thousand times. That loop exists on paper, but nobody runs it. A thousand small Gaussian steps collapse into one closed form draw:

x_t = sqrt(abar_t) * x_0 + sqrt(1 - abar_t) * eps

where eps is a fresh standard normal sample and abar_t is the cumulative product of the per step alphas, exposed in Diffusers as scheduler.alphas_cumprod. One index, one multiply, one add. Jumping straight to t = 731 costs exactly as much as jumping to t = 1.

That is what training looks like. You take a real latent, draw a random timestep, draw one noise tensor, and mix them. There is no sequential corruption anywhere in the training loop, so every sample in the batch can sit at a different noise level at no extra cost.

Read abar_t as a signal to noise dial. The signal variance is abar_t and the noise variance is 1 - abar_t, and they sum to 1, so the noisy sample keeps roughly unit variance at every t. On a typical scaled_linear schedule abar_0 is about 0.999 and abar_999 is about 0.005. At the top of the schedule your image contributes about half a percent of the variance. Almost nothing survives, and that matters later.

The score points back toward the data

Now flip the direction. To go backwards you need to know, for a given noisy sample, which way is uphill in probability. That is the score:

score(x_t) = grad_{x_t} log p_t(x_t)

A vector at every point in latent space, pointing toward denser data. You cannot compute p_t for real images, but you never have to. Under the noising equation above the conditional density is a Gaussian centred on sqrt(abar_t) * x_0, and its log gradient is closed form:

score(x_t) = -(x_t - sqrt(abar_t) * x_0) / (1 - abar_t) = -eps / sqrt(1 - abar_t)

The score is the noise you added, negated and rescaled. So a network trained to predict eps from x_t is a score estimator with a constant factor attached. This is denoising score matching, and it is the reason a plain mean squared error regression on noise gives you a sampler at all.

Once you have the score you can estimate the clean sample without touching the network again. Tweedie's formula says the posterior mean of x_0 given x_t is:

x0_hat = (x_t + (1 - abar_t) * score) / sqrt(abar_t)

Substitute the score and it reduces to (x_t - sqrt(1 - abar_t) * eps_hat) / sqrt(abar_t), which is the line inside every Diffusers step method.

Why a direction and not an image

If Tweedie gives you x0_hat in one shot, why iterate at all? Because x0_hat is a mean, not a sample. At t = 900 the set of clean images consistent with that noisy tensor is enormous, and the average of an enormous set of images is grey brown mush. Decode x0_hat at high t and you get exactly that. The model is not broken, it is being honest.

Sampling works because a small step in the score direction lands you somewhere less ambiguous, where the posterior mean is sharper, and so on. Ambiguity resolves gradually instead of all at once. The network predicts a direction because a direction composes across steps and an image does not.

There is a second reason, and it is about gradients. The eps target has unit variance at every timestep, so the loss magnitude is stable across the whole schedule. An x_0 target does not. That is why prediction_type="epsilon" beat prediction_type="sample" in practice, even though the two are algebraically interchangeable.

Velocity is the tangent, not a fourth idea

Epsilon prediction has a defect at the far end. At t = 999 the noisy sample is about 99.5 percent noise, so predicting eps is close to copying the input. The loss goes near zero and carries almost no information about the image. Predicting x_0 has the mirror defect at low t.

Write sqrt(abar_t) = cos(phi) and sqrt(1 - abar_t) = sin(phi), legal because the two square to 1. Then x_t = cos(phi) * x_0 + sin(phi) * eps is an arc, and differentiating along it gives:

v = d x_t / d phi = cos(phi) * eps - sin(phi) * x_0

which in scheduler terms is sqrt(abar_t) * eps - sqrt(1 - abar_t) * x_0. That is DDPMScheduler.get_velocity, verbatim. Velocity is not a third target invented for convenience, it is the tangent to the noising path, and it blends the two targets so neither endpoint degenerates.

Here is the whole parameterisation, checked numerically:

import torch
from diffusers import DDPMScheduler
 
sched = DDPMScheduler(
    num_train_timesteps=1000,
    beta_start=0.00085,
    beta_end=0.012,
    beta_schedule="scaled_linear",
    prediction_type="v_prediction",
)
 
torch.manual_seed(0)
x0 = torch.randn(1, 4, 64, 64)
eps = torch.randn(1, 4, 64, 64)
t = torch.tensor([500])
 
x_t = sched.add_noise(x0, eps, t)
v = sched.get_velocity(x0, eps, t)
 
abar = sched.alphas_cumprod[t]
a, s = abar.sqrt(), (1 - abar).sqrt()
 
print("x_t is a*x0 + s*eps :", torch.allclose(x_t, a * x0 + s * eps, atol=1e-5))
print("v   is a*eps - s*x0 :", torch.allclose(v, a * eps - s * x0, atol=1e-5))
print("x0 back from x_t, v :", torch.allclose(a * x_t - s * v, x0, atol=1e-3))
print("eps back from x_t, v:", torch.allclose(s * x_t + a * v, eps, atol=1e-3))

Four True. Given any two of x_0, eps, v at a known t, the third is a rotation away. A checkpoint's prediction_type is a choice about training signal, not about what the model knows.

Try it

Verify the score identity and Tweedie's formula across the schedule, then read the dial.

import torch
from diffusers import DDPMScheduler
 
sched = DDPMScheduler(
    num_train_timesteps=1000,
    beta_start=0.00085,
    beta_end=0.012,
    beta_schedule="scaled_linear",
)
 
torch.manual_seed(0)
x0 = torch.randn(1, 4, 64, 64)
eps = torch.randn(1, 4, 64, 64)
 
for t in [1, 200, 400, 600, 800, 999]:
    ti = torch.tensor([t])
    abar = sched.alphas_cumprod[ti]
    x_t = sched.add_noise(x0, eps, ti)
    score = -eps / (1 - abar).sqrt()
    x0_hat = (x_t + (1 - abar) * score) / abar.sqrt()
    print(t, round(float(abar), 5), torch.allclose(x0_hat, x0, atol=1e-3))
 
signal_half = (sched.alphas_cumprod < 0.5).nonzero()[0].item()
print("half signal at t =", signal_half)

You are done when every row prints True and you can state the timestep where signal and noise variance are equal for this schedule. Then change beta_end to 0.02 and rerun: that timestep should move earlier, because a heavier schedule destroys the image sooner.

Common mistakes

• Simulating the forward process as a loop. Applying a small noise step 500 times is slower and drifts from the closed form through accumulated float error. add_noise(x0, eps, t) is the definition, not an approximation of something else.
• Constructing a scheduler instead of loading the checkpoint's one. A v_prediction checkpoint driven by a default epsilon scheduler does not produce visible noise, it produces washed out, low contrast mush that reads as a bad prompt. Load with from_pretrained(repo, subfolder="scheduler") and print sched.config.prediction_type before you debug anything else.
• Reading x0_hat at high t as a preview of the output. It is a posterior mean over a huge set of images and it is supposed to look blurry at t = 900. Judge it around t = 200, not at the start.
• Dropping the minus sign on the score. score = -eps / sigma. With the sign flipped, every step pushes away from the data manifold and output degrades steadily instead of failing loudly.
• Expecting alphas_cumprod on a flow model. FlowMatchEulerDiscreteScheduler carries sigmas and a scale_noise method instead of alphas_cumprod and add_noise. Reaching for the wrong attribute is an AttributeError, which is the good case.

Where this goes next

Next session, Flow Matching Won, replaces the arc with a straight line between noise and data and shows why SD3.5, FLUX and Qwen-Image all train on that velocity instead of epsilon. The rotation you just verified is what makes that switch a change of path rather than a change of model.