SDXL-Lightning by ByteDance

[thiner]

A new base model, https://huggingface.co/ByteDance/SDXL-Lightning just released by ByteDance. They claimed it can generate high quality images in just 2 steps(even 1 step).

[mashb1t]

@thiner thanks for the info, this should be usable OOTB with overriding the sampling steps as well as cfg scale, similar to sdxl turbo!

[newxhy]

https://huggingface.co/ByteDance/SDXL-Lightning，Forge is already supported, how to set fooocus?

[mashb1t]

As mentioned in my last comment, by overriding parameters. See #1499 for more insights how turbo can be set up. I haven't tested lightning, but the setup should be similar with a few adjusted values you can find in their comfy flow/model page on HF.

[blckbx]

Been testing SDXL Lightning on Fooocus by importing the 8-step safetensor file and configuring CFG, steps, sampler and scheduler as described on huggingface.co. Works pretty good. Personally I prefer more steps for better quality over speedy generation.

For reference:

/models/checkpoints/sdxl_lightning_8step.safetensors

Base Model: sdxl_lightning_8step.safetensors
Guidance Scale: 2
Sampler: Euler
Scheduler: sgm_uniform
Forced Overwrite of Sampling Step: 8

[Irmagon]

I try all combinations parameters...
This preset works fine:

"default_model": "sdxl_lightning_8step.safetensors",
"default_image_number": 1,
"default_cfg_scale": 1.0,
"default_sample_sharpness": 2.0,
"default_sampler": "dpmpp_2m_sde_gpu",
"default_scheduler": "lcm",
"default_performance": "Speed",
"default_overwrite_step": 8,

[mashb1t]

for me euler and sgm_uniform worked best, as they are more true to life (less bloom + saturation) and introduce less artifacts (e.g. on skin) than dpmpp_2m_sde_gpu and lcm

[xhoxye]

Some third-party lightning models can also be tried, such as the following:
https://civitai.com/models/112902/dreamshaper-xl?modelVersionId=354657
https://civitai.com/models/43977?modelVersionId=355837
https://civitai.com/models/129666?modelVersionId=356366

[thiner]

Is there any chance to refactor the Extreme Speed mode, switch from lcm to lightning, and reduce the steps to 4?

[mashb1t]

We have to keep in mind that there's a difference between models snd performance. Performance can be changed for every model independently, so Extreme Performance can be set for different models of all kinds. Lightning is one specific series of model differing in trained steps, but still fixed to steps. LCM is a speed LoRA.
=> lightning model can't replace the universally applicable performance Extreme Speed

[poor7]

=> lightning model can't replace the universally applicable performance Extreme Speed

@mashb1t Lightning can. There are the same LoRAs, for example this one for 8 steps. I successfully use it with any model.
Example image: 8 steps Juggernaut XL

[mashb1t]

@poor7 ah, ok, then i didn't have the full picture. I'll test this on the weekend.

[thiner]

@mashb1t Check this out: https://hf-mirror.com/ByteDance/SDXL-Lightning/resolve/main/sdxl_lightning_4step_lora.safetensors. They provide Lora models as well.

[kexul]

=> lightning model can't replace the universally applicable performance Extreme Speed

@mashb1t Lightning can. There are the same LoRAs, for example this one for 8 steps. I successfully use it with any model. Example image: 8 steps Juggernaut XL

Would you mind share your setting, this is amazing!

[Green-li]

I have tried the 4 steps SDXL-Lightning LoRA on the fooocus default model and as well as the LCM on 4 steps.
Here are the images:
from left to right base model, +sdxl-lightning 4 steps lora, +lcm 4steps

[Green-li]

The LCM and SDXL-Lightning using 4 steps both cost 1s on my RTX 3060 GPU while the base model cost 22s using 30 steps.

[felixsanz]

I hope you don't consider this shameless spam... but I have written an article about SDXL-Lightning explaining how it works and made a lot of comparisons (steps, lora, vs turbo/lcmlora, etc).

I'm sure it will be useful!

https://www.felixsanz.dev/articles/sdxl-lightning-quick-look-and-comparison

[mashb1t]

Thank you for sharing, much appreciated!

[blckbx]

Wow great articles! Muchas gracias!

[mashb1t]

Please find the new performance mode proposal in #2415.
Looking forward to your feedback!

[VelocityRa]

Thanks!
How to use finetunes like DreamShaper XL Lightning? It performs better according to this etc.
I made a json for it (latest release, not using your PR) but not sure I have the correct settings picked.

[mashb1t]

"Lightning" performance adds the LoRA for SDXL Lightning, which can be used with every base model.
DreamShaper XL Lightning and all "Lightning" models have this exact lightning LoRA already merged, same principle as for LCM.
Generally speaking you can use every Lightning model with Fooocus, but please keep before mentioned facts in mind and do not use performance lightning with them, as this will apply the Lightning LoRA twice and produces worse results.
Feel free to choose and use every model based on SDXL you like. You may have to adjust settings such as steps override though.

Hope this helps in understanding the structure ans background better.

[VelocityRa]

Thank you! What you said seems right from my tests.

For anyone looking for a preset json, here's one for Dreamshaper XL Lighting, just pass --preset lightning to Foocus (and keep the setting at Speed).
lightning.json

[xhoxye]

lightning.json, it is recommended to change the name, if you use lightning, it may conflict with the 2.3.0 update

[BradKML]

@mashb1t thanks for all the work but is there SDXL-lightning settings for fast anime generation? Would like to see if it is viable on a laptop with embedded GPU

[mashb1t]

You can select the anime preset in combination with lightning performance