Skip to content

Dynamic LoRA adapters #176

@micmcg

Description

@micmcg

Do you have plans to add support for dynamically loading/unloading LoRA adapters as in the upstream vLLM?
https://docs.vllm.ai/en/stable/features/lora/#dynamically-serving-lora-adapters

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions