Without disabling smart memory the weights go all into dedicated GPU causing OOM

[NeverOccurs]

I run comfyui on a 4090. Below is my system and comfyui parameter. With dynamic vram, all weights go into dedicated VRAM and the system stutters, generation becomes extremely slow. After I disabled it everything seems to be fine. Shared GPU can be used and generation becomes normal. Could --disable-dynamic-vram be kept until everything is fixed?
[INFO] Total VRAM 24564 MB, total RAM 65277 MB
[INFO] pytorch version: 2.8.0+cu128
[INFO] xformers version: 0.0.32.post2
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 26110.0
[INFO] Using xformers attention
[INFO] Python version: 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]
[INFO] ComfyUI version: 0.24.0

Edit: I think I can confirm now the problem is mainly smart memory. Dynamic vram is fine. After disabling smart memory things are moving smoothly right now. And the performance is good.

[ado11231]

Keep --disable-dynamic-vram on. If the old behavior works smoothly, there's nothing wrong with just using it. If that doesnt work try this.

--reserve-vram tells ComfyUI to hold back a few GB so dynamic VRAM can't fill the card to the choking point. This keeps the feature's benefits while stopping the OOM.

If that doesn't fix it, fall back to --disable-dynamic-vram

[NeverOccurs]

Keep --disable-dynamic-vram on. If the old behavior works smoothly, there's nothing wrong with just using it. If that doesnt work try this.

--reserve-vram tells ComfyUI to hold back a few GB so dynamic VRAM can't fill the card to the choking point. This keeps the feature's benefits while stopping the OOM.

If that doesn't fix it, fall back to --disable-dynamic-vram

I did more retrys and some weird things happen. sometimes with --disable-dynamic-vram and --disable-smart-memory on things are cool and I can see shared GPU memory is used properly. Other times for example after I reboot my system, using same parameters, the model weights pour into dedicated GPU again causing system to stutter. No shared GPU is used again. Now at this point, with dynamic vram on everything is cool again. I am still investigating whether it is something with smart memory.

[NeverOccurs]

Keep --disable-dynamic-vram on. If the old behavior works smoothly, there's nothing wrong with just using it. If that doesnt work try this.

--reserve-vram tells ComfyUI to hold back a few GB so dynamic VRAM can't fill the card to the choking point. This keeps the feature's benefits while stopping the OOM.

If that doesn't fix it, fall back to --disable-dynamic-vram

I think I can confirm now the problem is mainly smart memory. Dynamic vram is fine. After disabling smart memory things are moving smoothly right now. And the performance is good.

[aimarxjg]

How do I disable Smart Memory? After updating to the August 1st version, I noticed high memory usage; when using the K2 model (Q4) with two LoRAs, it immediately starts using virtual memory, causing the generation time to jump to 900 seconds!

[NeverOccurs]

How do I disable Smart Memory? After updating to the August 1st version, I noticed high memory usage; when using the K2 model (Q4) with two LoRAs, it immediately starts using virtual memory, causing the generation time to jump to 900 seconds!

Just add --smart-memory into your .bat file. But i recently found that it might now be the culprit. You can try though.