A model that is a member of multiple groups

[LeBohdan]

Hello! Can you please suggest an idea, how to make one model a member of multiple groups? I need it to be launched at different times together with different models. Right now, I’ve solved the problem by simply creating a 'clone' (Nomic-embed-text-v2-moe_c).

Here is config:

"gemma3_tools_embed":
swap: false # all models from group allowed
exclusive: true #unload other groups

members:
  - "Gemma-3-27B-tools"
  - "Nomic-embed-text-v2-moe"

"continue":
swap: false # all models from group allowed
exclusive: true #unload other groups

members:
  - "Qwen3-Coder-30B-A3B-Instruct_Q5"
  - "Qwen2.5-coder-7B-instruct-Q4"
  - "Nomic-embed-text-v2-moe_c"

[mostlygeek]

You could put Nomic-embed-text-v2-moe into it's own group with persistent: true. Then it won't get swapped out. You'll have to manually unload it however.

[LeBohdan]

Thanks for the note. One more question: when setting up another model that requires full memory, do I need to manually unload the persistent model?

[mostlygeek]

Yes, you will have to.

FYI: I opened #304 where I've been gathering feedback for supporting a hook/plugin system for people to implement their own swapping strategies.