A way to eject the loaded model to save VRAM?

[giteeeeee]

I often play games while having a1111 still in the background. It eats ~6g of my vram while loaded with whichever checkpoint.

So is there a way to eject the loaded model somehow, without killing the process?

[w-e-w]

Settings > Actions > Unload SD checkpoint to RAM

[watzon]

This doesn't appear to exist anymore, but it is still the answer that shows up first in Google. Is there a new way to do this?

[w-e-w]

This doesn't appear to exist anymore

look again it's there


I recall there is some bugs with the system if you have multiple models loaded at once but it still does unload the current model

[randelreiss]

I've tried this feature. It doesn't appear to free up very much GPU RAM - maybe 10%? And then it frequently crashes AUTOMATIC1111 - I don't know if it's crashed on the web front-end side or something related to the GPU card - but I have to quit AUTOMATIC1111 and restart it to recover.

[andreiramani]

unload checkpoint to RAM, it will eats up RAM if we change model frequently. How to unload completely from RAM and VRAM?

[randelreiss]

Ollama framework has a really handy environment and API accessible variable:

OLLAMA_KEEP_ALIVE=[# of seconds] | [xM] | 0

I think it's mostly used for people who want the last loaded chat model to stay loaded longer. But I use it set to zero to keep the GPU VRAM as empty as possible as soon as possible. This is because I have many users that mostly use the GPU for chat and occasionally for Text-to-speech and SD image creation - loading up the GPU VRAM. Unfortunately SDWeb keeps its last model loaded indefinately. It would be great if SDWeb had a similar Keep Alive option to let us decide how long to keep the last model loaded.

[freakontrol]

I have the same problem using SDWeb as OpenWebUI integration.

[jhemmond]

Ollama framework has a really handy environment and API accessible variable:

OLLAMA_KEEP_ALIVE=[# of seconds] | [xM] | 0

I think it's mostly used for people who want the last loaded chat model to stay loaded longer. But I use it set to zero to keep the GPU VRAM as empty as possible as soon as possible. This is because I have many users that mostly use the GPU for chat and occasionally for Text-to-speech and SD image creation - loading up the GPU VRAM. Unfortunately SDWeb keeps its last model loaded indefinately. It would be great if SDWeb had a similar Keep Alive option to let us decide how long to keep the last model loaded.

Have you found the solution to this?

[randelreiss]

I haven't. I'm assuming the Stable Diffusion staff is chasing RTX 5xxx cards now with well over 16G of RAM. People like us sharing an old RTX 3060 with 12G among multiple users with Ollama & TTS support are the least of their interests.

[jhemmond]

I’m looking to hack it my own way. Will keep you posted on what I can do.

…

On Sun, Feb 16, 2025 at 6:35 PM, Randel Reiss ***@***.***(mailto:On Sun, Feb 16, 2025 at 6:35 PM, Randel Reiss <<a href=)> wrote: I haven't. I'm assuming the Stable Diffusion staff is chasing RTX 5xxx cards now with well over 16G of RAM. People like us sharing an old RTX 3060 with 12G among multiple users with Ollama & TTS support are the least of their interests. — Reply to this email directly, [view it on GitHub](#14449 (reply in thread)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AYIQWAYNANCSMUTRYH66ZLT2QEOEXAVCNFSM6AAAAABXIAWXJWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMRRHAZDSNA). You are receiving this because you commented.Message ID: ***@***.***>