Authorization Scheme - #198
Draft
AryanAhadinia wants to merge 6 commits into
Draft
Conversation
AryanAhadinia
force-pushed
the
feat/model-authorization-rebased
branch
from
August 7, 2026 13:05
9bcc4fd to
8e8950d
Compare
Contributor
|
I want to check we are not missing something from authentik here, I understand how you are implementing this but the auth isn't quite what we specified in the original requirements (won't link it here publicly) but agree this is a useful feature |
AryanAhadinia
marked this pull request as draft
August 8, 2026 12:01
…mission
Companion to serving-api's label-based model authorization (ADR-0001). A
launch now declares who the model is for, as an OpenTela peer label the
gateway enforces:
sml ... --authorization public # anyone (default)
sml ... --authorization private # only the launcher
sml ... --authorization user1@epfl.ch,user2@ethz.ch # only these users
"private" never reaches the mesh — the gateway cannot know who submitted a
job, so the CLI resolves it to the launcher's own email via the Serving
API's GET /v1/whoami before submission, and fails the launch rather than
falling back to public if that call can't be made. LaunchArgs rejects an
unresolved "private" as a guardrail for any path that forgets.
Values are canonicalized (strip, lowercase, dedupe) so two launches meaning
the same policy emit the same label, and a served name whose replicas
disagree on policy is refused by the gateway for everyone — so a launch
that would collide with a running model under a different policy is refused
up front instead of taking that model down with it.
Applied on every path that submits a launch: `preconfigured`, `advanced`,
both `loadtest` launch paths, and the MCP server's launch tool.
SonarCloud's quality gate failed the PR on new-code coverage (65.7% vs the 80% threshold); every other condition passed. The gap was serving_api.py, whose two HTTP calls had no tests at all, plus the prompt validator. Covers whoami and served_model_authorizations against a stubbed httpx client: success, 401, other non-2xx, transport error, timeout, and the empty-payload cases. These decide who can reach a model, so the point of most of them is that a failure surfaces as ServingApiError rather than a silent empty answer a caller could read as "no restriction". Also drives `sml advanced` end to end for the first time, asserting that --authorization private becomes the launcher's email and that the conflict check is asked about the namespaced served name.
Served names are <username>/<vendor>/<model>, but every example passed the bare <vendor>/<model> and relied on SML prepending the account. Spell the namespace out instead — 100 occurrences across 88 shell examples become "$USER/<vendor>/<model>", and the four Python examples move from the old "-<username>" suffix to a real namespace prefix. SML still prepends a missing namespace, so existing user scripts keep working; the examples just no longer depend on it. $USER has to be the *cluster* account. It normally is, but running sml from a laptop with a different local username now fails the namespace check instead of silently publishing under the wrong name — documented in usage-advanced, with the escape hatches (explicit name, or omit it). Tests follow the same rule: - the unit example test expands $USER the way a shell would before parsing, since shlex does not; - the CLI example integration test runs each script with USER pinned to the account FirecREST reports, because a CI runner's $USER is not the cluster account, and asserts the launched name is namespaced under it; - the launcher integration test now names the model explicitly rather than exercising the prepend path. Also corrects served_name.py and the glossary, which claimed the gateway cross-checks the namespace against launched_by. It does not — it only requires three non-empty segments — so the check is ours and is a collision guard, not an ownership guarantee.
Dropping the salt made a derived name a pure function of user + model, so one user launching a model on sglang and on vllm advertised both under the same id. OpenTela then load-balances across two different engines under one name, and CI hits it head-on: the lightweight suite launches Apertus-8B-Instruct-2509 on both frameworks, in parallel. Names SML derives are now <username>/<vendor>/<model>-<framework>. Explicit --served-model-name is untouched: if a user picks a name, it is theirs, even if that means two frameworks share it. Two launches of one model on one framework still share a name — those are replicas, which is the point. derive_served_model_name is the single rule, used by preconfigured, the MCP tool, and both launchers' fallback. The suffix keeps the id at three segments, which is all the gateway will list. The integration test names each launch explicitly and now includes the topology too: the std matrix runs one model+framework in three configurations concurrently, which the framework suffix alone would not separate. All 15 CI launches get distinct names.
The previous runs failed in the FirecREST integration tests with cluster-side errors on clariden (503 scheduler unhealthy, then 408 command timeout), not with anything the branch changed. Re-running to see whether the cluster has recovered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds an OpenTela peer label `sml_version`, so a running model on the mesh says which SML produced it. That is the first thing worth knowing when a launch misbehaves — whether it predates a fix — and until now it was only in the telemetry payload, which the gateway never sees. Resolved on the submitting machine and baked into the script as a literal: SML is not installed on the compute node, so a $(...) would render empty, and the version that matters is the one that built the launch. The lookup moves into _sml_version(), shared with the telemetry payload so the two cannot drift, and falls back to "unknown" rather than raising when the package metadata is missing — an uninstalled source checkout should not fail a launch over a provenance label. No gateway change needed: /v1/models_detailed already passes peer labels through verbatim, so the value shows up per model as labels.sml_version. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AryanAhadinia
force-pushed
the
feat/model-authorization-rebased
branch
from
August 11, 2026 21:25
4416b3b to
7a9b948
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…mission
Companion to serving-api's label-based model authorization (ADR-0001). A launch now declares who the model is for, as an OpenTela peer label the gateway enforces:
"private" never reaches the mesh — the gateway cannot know who submitted a job, so the CLI resolves it to the launcher's own email via the Serving API's GET /v1/whoami before submission, and fails the launch rather than falling back to public if that call can't be made. LaunchArgs rejects an unresolved "private" as a guardrail for any path that forgets.
Values are canonicalized (strip, lowercase, dedupe) so two launches meaning the same policy emit the same label, and a served name whose replicas disagree on policy is refused by the gateway for everyone — so a launch that would collide with a running model under a different policy is refused up front instead of taking that model down with it.
Applied on every path that submits a launch:
preconfigured,advanced, bothloadtestlaunch paths, and the MCP server's launch tool.