VRAM Overflow.

[dima-j]

I have GTX 3060 12gb, drivers 546.33.
When generating images larger than 1024x1024 using SDXL models, the video memory overflows at the end. Because of this, the final stages of generation slow down very much. Up to 97% of generation VRAM consumption stable 6.5-8 GB. But at the last stage, consumption reaches 13-15 gb. RAM begins to be used and generation slows down very much. Is there a way to solve this problem?

[TKoestlerx]

The peak at the end is probably when the VAE is decoding the final Image.

There are two easy things to reduce the VRAM Usage in this case.

a) Start with the CommandLine Option "--medvram-sdxl"
With this Option the SDXL Model is split in Text-Encoder,Unet and VAE. And only one is active at any given Time. So it will release the Memory used by the unet and the Text-Encoder when the VAE is doing its job.

b) Use a 16-bit VAE.
https://huggingface.co/madebyollin/sdxl-vae-fp16-fix can reduce the Memory Usage of the VAE by half.

With both Options i can generate 1280x1280 with a batch-size of 4 on the same card you have.