Some questions about training.

[weilu4606]

Hello, regarding the flux dev distillation model, how should the guidance_scale be set during fine-tuning? And how should the guidance_scale be set for both the controlnet and transformer when training the controlnet?

[EndlessSora]

Hi @weilu4606 , thanks for your interest!
During training, please set guidance_scale = 1 for both transformer (frozen) and controlnet to align with no CFG status.
During inference, please set guidance_scale > 1 (e.g., 3.5) for transformer and guidance_scale = 1 for controlnet.

#45

[weilu4606]

Dear author, thank you for your reply. Does the resampler need to be trained separately, or can it be trained end-to-end with ControlNet? Does training the resampler require images of relatively large faces? I would be very grateful for any answers you can provide.

[EndlessSora]

@weilu4606 The resampler can be trained end-to-end. It does not require large faces only.