About ControlNet train with the 1step PixArt-DMD in diffusers.

[CuddleSabe]

Hi, I found that the 1step model PixArt-DMD is supported to the diffusers. So I write a train script in diffusers,
the output is chunk to two components: pred_noise and vb,
I only use pred_noise to compute mse loss, the loss decrease very fast, but the output image is very bad
what should I do?

[CuddleSabe]

add up to 256 batch size with 8 a100.

[lawrence-cj]

We released here: https://github.com/PixArt-alpha/PixArt-sigma/blob/master/asset/docs/pixart_dmd.md

[CuddleSabe]

hi, I found when I "with torch.autocast("cuda"):" for the diffusers PixArt pipeline,

the output will like this