-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Labels
enhancementNew feature or requestNew feature or request
Description
NOTE: this is just food for thought, it's kind of a non-trivial feature from the user's perspective.
OpenAI has the v1/models/* endpoints, see here. We might make use of these as follows (also serves for the purpose of OpenAI compat #3)
- GET
/v1/models/listlists the supported models - GET
/v1/models/{model}prepares the topology for the given model, akin to callingprepare_topology- we would need some extra request parameters (such as
strategy, but we could embed them into the{model}param too, e.g.ring:gpt-oss-20b); bit smelly though
- we would need some extra request parameters (such as
- DELETE
/v1/models/unloads the model, akin to callingunload_modelon API.
The idea here is to allow the entire user-flow of dnet to be achievable with openai compatibility, which enables us to use all existing OpenAI-compatible clients out there.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request