How to pick a sampler method?

Image Convergence

In this section, I will generate the same image using different samplers with up to 40 sampling steps. The last image at the 40th step is used as a reference for evaluating how quickly the sampling converges. The Euler method will be used as the reference.

Euler, DDIM, PLMS, LMS Karras and Heun

First, let’s look at the Euler, DDIM, PLMS, LMS Karras, and Heun as a group since they represent old-school ODE solvers or original diffusion solvers. DDIM converges at about the steps as Euler but with more variations. This is because it injects random noise during its sampling steps.

Image convergence of Euler, DDIM, PLMS, LMS Karras and Heun (Lower is better).

PLMS did not fare very well in this test.

LMS Karras seems to have difficulty converging and has stabilized at a higher baseline.

Heun converges faster but is two times slower since it is a 2nd order method. So we should compare Heun at 30 steps with Euler at 15 steps, for example.

Ancestral samplers

If a stable, reproducible image is your goal, you should not use ancestral samplers. All the ancestral samplers do not converge.

Ancestral samplers do not converge well (Lower is better).

DPM and DPM2

DPM fast did not converge well. DPM2 and DPM2 Karras perform better than Euler but again in the expense of being two times slower.

DPM adaptive performs deceptively well because it uses its own adaptive sampling steps. It can be very slow. DPM adaptive is available in NovArch's premium plans only.

Convergence of DPM samplers (Lower the better).

DPM++ solvers

DPM++ SDE and DPM++ SDE Karras suffer the same shortcomings as ancestral samplers. They not only don’t converge, but the images also fluctuate significantly as the number of steps changes.

DPM++ 2M and DPM++ 2M Karras perform well. The Karras variant converges faster when the number of steps is high enough.

Convergence of DPM++ samplers (Lower the better).

UniPC

UniPC converges a bit slower than Euler, but not too bad.

 UniPC sampler compared to Euler

Speed

Relative rending time of each method (lower the better)

Although DPM adaptive performs well in convergence, it is also the slowest.

You may have noticed the rest of the rendering times fall into two groups, with the first group taking about the same time (about 1x), and the other group taking about twice as long (about 2x). This reflects the order of the solvers. 2nd order solvers, although more accurate, need to evaluate the denoising U-Net twice. So they are 2x slower.

Quality

Of course, speed and convergence mean nothing if the images look crappy.

Final images

Let’s first look at samples of the image.

So… which one is the best?

Here are my recommendations:

1. If you want to use something fast, converging, new, and with decent quality, excellent choices are

   • DPM++ 2M Karras with 20 – 30 steps

   • UniPC with 20-30 steps.

2. If you want good-quality images and don’t care about convergence, good choices are

   • DPM++ SDE Karras with 10-15 steps (Note: This is a slower sampler)

   • DDIM with 10-15 steps.

3. Avoid using any ancestral samplers if you prefer stable, reproducible images.

4. Euler and Heun are fine choices if you prefer something simple. Reduce the number of steps for Heun to save time.