Skip to content

feat: piggybacking OpenAI endpoints #46

@erhant

Description

@erhant

NOTE: this is just food for thought, it's kind of a non-trivial feature from the user's perspective.

OpenAI has the v1/models/* endpoints, see here. We might make use of these as follows (also serves for the purpose of OpenAI compat #3)

  • GET /v1/models/list lists the supported models
  • GET /v1/models/{model} prepares the topology for the given model, akin to calling prepare_topology
    • we would need some extra request parameters (such as strategy, but we could embed them into the {model} param too, e.g. ring:gpt-oss-20b); bit smelly though
  • DELETE /v1/models/ unloads the model, akin to calling unload_model on API.

The idea here is to allow the entire user-flow of dnet to be achievable with openai compatibility, which enables us to use all existing OpenAI-compatible clients out there.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions