Skip to content

Authorization Scheme - #115

Open
AryanAhadinia wants to merge 9 commits into
mainfrom
feat/model-authorization-rebased
Open

Authorization Scheme#115
AryanAhadinia wants to merge 9 commits into
mainfrom
feat/model-authorization-rebased

Conversation

@AryanAhadinia

@AryanAhadinia AryanAhadinia commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This PR implements an authorization scheme for accessing the served models in the serving platform. The models should provide an OpenTela --authorization label. This label may accept two values:

  • --authorization public. The model is publicly available for all of the users in the platform.
  • --authorization email_1@epfl.ch,...,email_n@ethz.ch. Authorizing a group of users to access the model.

For maintaining backward-compatibility, we default to --authorization public if a label is not provided.

Note that in SML, we also provide --authorization private option. However, this will be translated to group authorization before reaching the mesh using whoami endpoint which is introduced in this PR.

whoami endpoint will return the email of the user associated with the API key sent with the request. This is necessary to translate the serving API key to the email address of the user.

This PR is coupled with swiss-ai/model-launch#198, however they are not planned to merged together. The merging plan would be as below.

  • Stage 0: Review serving-api PR.
  • Stage 1: Merging the serving-api PR and deploy to dev environment.
  • Stage 2: Test and review SML PR.
  • Stage 4: Merge SML PR to main and create a serving-api release to deploy to prod env.

This PR is related to the FR-LAUN-CH-## codes of the requirements of the serving platform. Details will be discussed privately in Slack.

A by-product of this PR is the sml_version label, which can be later used to enforce min sml version. The implementation of this enforcement is also provided, but is disabled for now.

@AryanAhadinia
AryanAhadinia force-pushed the feat/model-authorization-rebased branch from 2464c8f to 1fb6eef Compare August 7, 2026 13:05
@AryanAhadinia
AryanAhadinia force-pushed the feat/model-authorization-rebased branch 2 times, most recently from e9a0041 to b663909 Compare August 8, 2026 14:58
FarukZahiragic
FarukZahiragic approved these changes Aug 9, 2026
View reviewed changes
Comment thread backend/services/model_service.py
def _sml_version_exclusion(model: dict) -> str | None:
"""Why this launch's SML version disqualifies it — None when it passes.

**Inert until MIN_SML_VERSION is set**, which it is not by default: with

@FarukZahiragic FarukZahiragic Aug 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what is this MIN_SML_VERSION value going to be, are we gonna be changing it dynamically? Is it the same for all models?

@AryanAhadinia AryanAhadinia Aug 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this value is set, we will drop the models which are launched by SML below that version. It is automatically passed by the SML as a label to the OpenTela fabric.

robmsmt
robmsmt reviewed Aug 10, 2026
View reviewed changes
Comment thread backend/services/auth_service.py Outdated
AryanAhadinia and others added 9 commits August 13, 2026 22:55
@AryanAhadinia @claude
feat(authorization): per-user model authorization via OpenTela labels
203a91e 
Models launched via SML now carry an "authorization" peer label: "public"
or a comma-separated email list (SML resolves its "private" default to
the launcher's email before submission). The backend enforces it and the
frontend surfaces it:

- GET /v1/whoami: resolve the caller's API key to their email, so SML
  can translate --authorization private before launching.
- /v1/models and /v1/models_detailed take an optional bearer: anonymous
  callers see only public models, a valid API key adds the models whose
  authorization list includes the caller, an unknown key gets 401.
- Every inference proxy route (chat/completions, completions, responses,
  embeddings, rerank, score, classify, tokenize, detokenize) checks the
  model's authorization before proxying and returns 403 in the OpenAI
  permission_error envelope when the caller is not listed.
- A missing label means public, so every pre-feature launch keeps
  working; passthrough providers (CSCS L1, RCP) stay public. The DNT-
  derived authorization map is cached ~10s, serves stale on fetch
  failure, and fails open (logged) only at true cold start.
- Frontend: the model list authenticates with the stored API key
  (falling back to session -> /v1/profile), retries anonymously on 401,
  and restricted models get a "Restricted" badge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA7PtcMPSV6cim7TARFE54
@AryanAhadinia @claude
docs(adr): record the label-based model authorization decision
09d0c10 
ADR-0001 captures why authorization is a launch-time OpenTela label with
client-side resolution of "private" (via /v1/whoami), the missing-label=
public back-compat rule, the fail-open-at-cold-start policy, and the
alternatives rejected (DB ACLs, server-side resolution, fail-closed,
frontend-only filtering).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA7PtcMPSV6cim7TARFE54
@AryanAhadinia @claude
fix(authorization): refuse to route served-name collisions with confl…
00fb055 
…icting labels

Independent launches can advertise the same served model name with
different authorization labels. Enforcement previously allowed a request
if ANY peer entry granted the caller — exploitable, because OpenTela
load-balances a name across every peer advertising it: a second launch
squatting a restricted model's name with authorization=public would
widen access to the restricted replicas, and any allow-on-collision can
route a caller's prompts to a replica they never chose to trust.

Labels now normalize to policies (public, or an email set — order, case,
spacing, and public-vs-missing never differ). One shared policy across
all entries applies as before, so replicas, followers, and consecutive-
chain handovers are unaffected. Distinct policies are a conflict: the
gateway refuses to route the name for everyone (403 naming the
conflict) until the collision resolves — union widens under attacker
control, and intersection still routes prompts to untrusted replicas.
Listing stays per-entry; the model card shows an "Auth conflict" badge
using the same normalization. ADR-0001 records the reasoning, including
the self-asserted-labels trust boundary this cannot close.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA7PtcMPSV6cim7TARFE54
@AryanAhadinia
fix(authorization): reconcile the label gate with namespaced model ids
f889e66 
main gained two things after this branch was written: provider-prefix
namespacing (resolve_provider -> resolve_model, ResolvedModel) and the
platform_namespaced listing filter. Reconcile against both.

- ensure_model_access resolves through resolve_model. A passthrough
  provider id stays public; SwissAI-Research/<org>/<model> is an alias
  the proxy strips before forwarding, so BOTH the prefixed and the
  upstream id are looked up and their labels pooled — otherwise the
  alias would route around a restricted model's policy.
- /v1/models* keeps platform_namespaced ahead of the per-caller filter,
  so an entry must be advertisable before it is authorization-checked.
- The isinstance guard for raw body values moves onto resolve_model.
- Tests: patch resolve_model, namespace the listing fixtures (an
  un-namespaced id is now dropped before authorization is consulted),
  expect provider-prefixed passthrough ids, and match llm_proxy's
  provider_label signature.
lic. The DNT-
  derived authorization map is cached ~10s, serves stale on fetch
  failure, and fails open (logged) only at true cold start.
- Frontend: the model list authenticates with the stored API key
  (falling back to session -> /v1/profile), retries anonymously on 401,
  and restricted models get a "Restricted" badge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA7PtcMPSV6cim7TARFE54

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# interactive rebase in progress; onto e758aa2
# Last command done (1 command done):
#    pick 6163065 feat(authorization): per-user model authorization via OpenTela labels
# Next commands to do (2 remaining commands):
#    pick 5d1efce docs(adr): record the label-based model authorization decision
#    pick 97f6d24 fix(authorization): refuse to route served-name collisions with conflicting labels
# You are currently rebasing branch 'feat/model-authorization-rebased' on 'e758aa2'.
#
# Changes to be committed:
#	modified:   README.md
#	modified:   backend/routers/classify.py
#	modified:   backend/routers/completions.py
#	modified:   backend/routers/embeddings.py
#	modified:   backend/routers/models.py
#	modified:   backend/routers/profile.py
#	modified:   backend/routers/rerank.py
#	modified:   backend/routers/responses.py
#	modified:   backend/routers/tokenization.py
#	modified:   backend/services/auth_service.py
#	new file:   backend/services/authorization_service.py
#	modified:   backend/services/model_service.py
#	modified:   backend/services/passthrough_service.py
#	modified:   backend/tests/test_app.py
#	new file:   backend/tests/test_authorization_routes.py
#	new file:   backend/tests/test_authorization_service.py
#	modified:   frontend/src/components/ui/ModelCard.svelte
#	modified:   frontend/src/components/ui/ModelList.svelte
#
@AryanAhadinia
docs(authorization): default to public, document the alias lookup
f3d780b 
The spec for this feature is public-by-default: restricting a model is an
explicit act, so enabling authorization can never make a model that used to
be visible disappear. ADR-0001 and the README described an SML default of
private; correct both.

Also record that a SwissAI-Research/ alias is checked against the upstream
id as well as the prefixed one, so the alias cannot route around a policy.
efixed and the
  upstream id are looked up and their labels pooled — otherwise the
  alias would route around a restricted model's policy.
- /v1/models* keeps platform_namespaced ahead of the per-caller filter,
  so an entry must be advertisable before it is authorization-checked.
- The isinstance guard for raw body values moves onto resolve_model.
- Tests: patch resolve_model, namespace the listing fixtures (an
  un-namespaced id is now dropped before authorization is consulted),
  expect provider-prefixed passthrough ids, and match llm_proxy's
  provider_label signature.
lic. The DNT-
  derived authorization map is cached ~10s, serves stale on fetch
  failure, and fails open (logged) only at true cold start.
- Frontend: the model list authenticates with the stored API key
  (falling back to session -> /v1/profile), retries anonymously on 401,
  and restricted models get a "Restricted" badge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UA7PtcMPSV6cim7TARFE54

# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# interactive rebase in progress; onto e758aa2
# Last command done (1 command done):
#    pick 6163065 feat(authorization): per-user model authorization via OpenTela labels
# Next commands to do (2 remaining commands):
#    pick 5d1efce docs(adr): record the label-based model authorization decision
#    pick 97f6d24 fix(authorization): refuse to route served-name collisions with conflicting labels
# You are currently rebasing branch 'feat/model-authorization-rebased' on 'e758aa2'.
#
# Changes to be committed:
#	modified:   README.md
#	modified:   backend/routers/classify.py
#	modified:   backend/routers/completions.py
#	modified:   backend/routers/embeddings.py
#	modified:   backend/routers/models.py
#	modified:   backend/routers/profile.py
#	modified:   backend/routers/rerank.py
#	modified:   backend/routers/responses.py
#	modified:   backend/routers/tokenization.py
#	modified:   backend/services/auth_service.py
#	new file:   backend/services/authorization_service.py
#	modified:   backend/services/model_service.py
#	modified:   backend/services/passthrough_service.py
#	modified:   backend/tests/test_app.py
#	new file:   backend/tests/test_authorization_routes.py
#	new file:   backend/tests/test_authorization_service.py
#	modified:   frontend/src/components/ui/ModelCard.svelte
#	modified:   frontend/src/components/ui/ModelList.svelte
#
@AryanAhadinia
fix(frontend): resolve the models bearer without caching the API key
ce3936f 
Rebasing onto main brought in #112, which removed API-key caching from
localStorage: that store is per-browser, not per-account, so a cached key
outlives an account switch and shows the previous account's key. The
models list — added on this branch to send a bearer so restricted models
appear — still read and re-populated that cache, which would have
reintroduced the leak in a worse place: it decides which *models* you can
see, so a stale key lists another account's restricted models.

The bearer now always comes from /v1/profile for the current session, and
a mid-flight rotation (401) falls back to the anonymous public list.
@AryanAhadinia @claude
feat(models): surface sml_version, and add an inert minimum-version gate
b08a42c 
SML now labels each peer with the version that rendered its launch. Two
things follow, one live and one deliberately not.

Live: sml_version is pulled up next to launched_by in _peer_metadata and
carried on /v1/admin/models rows, with an SML column on the admin
all-models page. The value was already readable inside labels; this makes
it legible where "why is my model like this?" gets answered.

Not live: _sml_version_exclusion implements a MIN_SML_VERSION floor for
listing, mirroring MIN_USER_OTELA_VERSION — but the setting defaults to
empty, and empty means the check does not run. Nothing is excluded for it
today; enabling it later is a config change rather than a code change.
Tests pin that both ways: inert with no minimum (including for peers whose
label is missing or "unknown"), enforcing once one is set.

Platform launches are exempt by construction: they come from k8s, not SML,
so they carry no sml_version and would otherwise all disappear the moment a
minimum was configured. The otela reason still wins when both fail — it is
the more actionable of the two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@AryanAhadinia @claude
fix(authorization): check the id that actually gets routed
3ef72d6 
main's #122 stopped stripping the SwissAI-Research/ prefix — the full
prefixed id is the k8s deployment's served name, so it is forwarded
verbatim. The authorization gate was written against the old behaviour and
looked the policy up under both the prefixed and the stripped id, with a
docstring explaining a rewrite that no longer happens.

Look it up under the requested id, and pool the upstream id only when the
proxy actually rewrites it. That is a no-op today and keeps the alias from
becoming a way around a policy if id rewriting ever returns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@AryanAhadinia
Use Redis Caching
7a7ff52
@AryanAhadinia
AryanAhadinia requested a review from robmsmt August 13, 2026 20:56
@AryanAhadinia
AryanAhadinia force-pushed the feat/model-authorization-rebased branch from fdba553 to 7a7ff52 Compare August 13, 2026 21:01
@AryanAhadinia AryanAhadinia self-assigned this Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FarukZahiragic FarukZahiragic FarukZahiragic approved these changes
@robmsmt robmsmt Awaiting requested review from robmsmt

Assignees

@AryanAhadinia AryanAhadinia

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AryanAhadinia @robmsmt @FarukZahiragic