Why the DDIM scheduler does not work in diffusers?

[quattrinifabio]

I am trying to change the scheduler for inference, from the default DPMSolverMultistepScheduler to DDIM.
It should be enough to use the code pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config), but the results are very bad in quality. Anyone can think of a reason? The schedulers should all work during inference.

I have read PixArt-alpha/PixArt-sigma#65, where it is stated that the cause could be the advancement of DPMSolver. Though, in the paper it is stated that the results should be similar (page 12, appendix A.5)

[lawrence-cj]

You may find that we can also generate images with IDDPM here:

PixArt-alpha/scripts/interface.py

Line 85 in 4f8afe3

if sampler == 'iddpm':

and if you set the order of DPMSolver to 1, it will be the same as DDIM here:

PixArt-alpha/scripts/interface.py

Line 110 in 4f8afe3

order=2,

Both can get normal results. BTW, what is the sampling step you are using for DDIM?

[quattrinifabio]

Thank you for your answer!
I have tried your code, and it indeed works. My question was about the diffusers pipeline integration. I have difficulties making DDIM work there. I have tried with 20, 30, and 50 steps.

[danielajisafe]

In addition, the generations from Pixart using DPMS has low positional variation. They are mostly constrained to the center of the image for different noise/seed, using the same prompt.

DDIM, which the original poster was asking about, has much diversity for the same prompt; however, the quality is bad.

@lawrence-cj please do you have any comments on this?