[Feature request] Never automatically unload models

[nomonkeynodeal]

The basic function of llama-swap is to swap models, but it's more than that now. It provides a really nice web UI to load and unload without copy-pasting commands. Also a very under-appreciated pre-made docker, better than Ollama. I have not found anything with comparable functionality.

The automatic unloading every single time I accidentally make a call to the wrong model is major pain point. Loading a model on Strix Halo can take 1-2 minutes.

Getting a helper model to simply load at the same time as another model is always a fight. The groups are borderline impossible to understand. (I took one look at the Matrix letter jumbles and my brain rejected the entire scheme. I know from experience this is never going to happen.)

The two configs in one file paradigm (a second list of models in a group config) is very difficult to manage. I am changing the models that I use every day. It's hard enough getting each individual entry right without managing a second list, scrolling up and down, manually syncing model names.

Maybe there should just be a simple way to stop the constant unloading of models and allow simple manual operation?

[knguyen298]

I, too, don't use the matrix feature and have no plans to try to; with all due respect to mostlygeek it seems unintuitive and needlessly complex.

However, I feel like this is already solved with the groups feature, which has unfortunately has had its example/documentation removed from the config example.

---

In my config, I have two models I always want preloaded and never unloaded: task-model and text-embedding. I have a root key called groups configured as such:

groups:
    ...
    "Persistent Models":
    swap: false
    exclusive: false
    persistent: true
    members:
      - "text-embedding"
      - "task-model"

These models will never unload, and I'm free to load and unload other models at will. It doesn't matter if there aren't any other groups either.

If I ever switch the underlying models used, I just update the entry for the model in the config, leaving the groups section untouched. For example, if I want to switch from Qwen3.5 0.8B to a hypothetical Qwen3.6 0.8B, I just go from:

models:
  "task-model":
    cmd: |
      ${llama}
      --model /models/Qwen3.5-0.8B-Q8_0.gguf
      ...

to

models:
  "task-model":
    cmd: |
      ${llama}
      --model /models/Qwen3.6-0.8B-Q8_0.gguf
      ...

[nomonkeynodeal]

POV: You just deleted some llama-swap entries for gemma-4-31B-it-Q5_K_L_spec_decode, gemma-4-31b-unslop-good-lora-v2-Q4_K_M, Gemma-4-Garnet-31B-heretic-Q5_K_M. Now scroll down 6 pages and delete only those entries from the second group config at the bottom of the file. DO NOT delete gemma-4-31B-it-uncensored-heretic-Q5_K_M or gemma-4-31B-it-Q5_K_L_spec_decode_E2B. Oh you forgot which ones you were deleting? Get ready, it's doing to be a long journey back to the top. Hope you remember this time!

This whole exercise is necessary just to stop models from randomly unloading when a stray call is made. I don't think I've ever intentionally swapped a model via an API call. Honestly, I don't even see how that could be useful given it takes like at least 20 seconds even for a small 31B. Most of mine take like a minute to load. By the time the model is switched the event that would require the model is long passed. I guess people want to automatically chat with a different model? I don't know.

[mostlygeek]

I, too, don't use the matrix feature and have no plans to try to; with all due respect to mostlygeek it seems unintuitive and needlessly complex.

It's contextual depending on your specific needs. For some use cases the swap matrix made them possible.

However, I feel like this is already solved with the groups feature, which has unfortunately has had its example/documentation removed from the config example.

When the project started the single config.yaml example was sufficient documentation. Now it's nearly impenentrable by new and even experienced users. I have some plans to document things better.

[Xnyle]

@mostlygeek actually I looked at the matrix and immediately felt at home, no clue what's complicated about that. Whatever you change please don't abandon that concept. I didn't even realize that the is a groups feature and I don't get why it would be needed as matrix can handle all use cases?