Euler Sampler

The inner working of these samplers is quite mathematical in nature. We will only explain Euler (The simplest one) in detail. Many of them share elements of Euler.

Euler

Euler is the most straightforward sampler possible. It is mathematically identical to Euler’s method for solving ordinary differential equations. It is entirely deterministic, meaning no random noise is added during sampling.

Below is the sampling step-by-step.

Step 1: Noise predictor estimates the noise image from the latent image.

Step 2: Calculate the amount of noise needed to be subtracted according to the noise schedule. That is the difference in noise between the current and the next step.

Step 3: Subtract the latent image by the normalized noise image (from step 1) multiplied by the amount of noise to be reduced (from step 2).

Repeat steps 1 to 3 until the end of the noise schedule.

Noise schedule

But how do you know the amount of noise in each step? Actually, this is something you tell the sampler.

A noise schedule tells the sampler how much noise there should be at each step. Why does the model need this information? The noise predictor estimates the noise in the latent image based on the total amount of noise supposed to be there. (This is how it was trained.)

A noise schedule for 15 sampling steps.

There’s the highest amount of noise in the first step. The noise gradually decreases and is down to zero at the last step.

Changing the number of sampling steps changes the noise schedule. Effectively, the noise schedule gets smoother. A higher number of sampling steps has a smaller reduction in noise between any two steps. This helps to reduce truncation errors.

From random to deterministic sampling

Do you wonder why you can solve a random sampling problem with a deterministic ODE solver? This is called the probability flow formulation. Instead of solving for how a sample evolves, you solve for the evolution of its probability distribution. This is the same as solving for the probability distribution instead of sample trajectories in a stochastic process.

Compared with a drift process, these ODE solvers use the following mappings.

• Time → noise

• Time quantization → noise schedule

• Position → latent image

• Velocity → Predicted noise

• Initial position → Initial random latent image

• Final position → Final clear latent image

Sampling example

Below is an example of text-to-image using Euler’s method. The noise schedule dictates the noise level in each step. The sampler’s job is to reduce the noise by just the right amount in each step to match the noise schedule until it is zero at the last step.

Denoising with Euler’s method.