fix(routing): 404 model ids without the canonical 3-segment namespace - #127
Merged
Conversation
Every routable id on the platform is <namespace>/<model_org>/<model_name> (SwissAI-Research/..., CSCS-Inference/..., RCP-AIaaS/..., or a username for user launches). Ids with any other shape — notably historical bare upstream ids like swiss-ai/Apertus-8B-Instruct-2509 — used to be silently claimed by the first registered passthrough provider that advertised them, with registration order picking the winner. Enforce the shape at the request boundary instead: a new require_namespaced_model helper 404s malformed ids (with the expected form in the message and a warning log for operators) across chat completions, completions, responses, embeddings, rerank, score, classify, tokenize and detokenize. The bare-id back-compat loop in resolve_model is now unreachable and removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Anyone sending a model id that is not exactly
<namespace>/<model_org>/<model_name>now gets a 404 at the request boundary, before any routing decision. Previously, bare upstream ids likeswiss-ai/Apertus-8B-Instruct-2509were silently claimed by the first registered passthrough provider that advertised them (CSCS L1 today) — which provider won was an accident of registration order.backend/middleware/model_id.py→require_namespaced_model(): accepts exactly three non-empty slash-separated segments (SwissAI-Research/...,CSCS-Inference/...,RCP-AIaaS/..., or<username>/...user launches); anything else 404s with the expected form in the error message, and logs a warning so operators can spot unmigrated clients./v1/chat/completions,/v1/completions,/v1/responses,/v1/embeddings,/v1/rerank,/v1/score,/v1/classify,/v1/tokenize,/v1/detokenize.resolve_modelis now unreachable and removed — no more silent first-provider-wins rerouting.Supersedes #125 (that PR removed the rerouting inside
resolve_modelonly; this enforces the invariant for all traffic at the boundary). Recommend closing #125 if this lands.Client impact
swiss-ai/Apertus-*form) and ids with missing/extra segments now 404 with:Model '...' not found. Model ids must be fully namespaced as <namespace>/<model_org>/<model_name> ....modelentirely also get this 404 instead of an opaque upstream error.CSCS-Inference/glm-4.5), its prefixed form has only 2 segments and would be listed in/v1/modelsbut unroutable. Current CSCS/RCP catalogs use HF-styleorg/nameids, so today's models are all fine.Tests
test_model_id_validation.py: valid/invalid shape matrix for the helper + a router-level test proving a bare id 404s before_resolve_routeis ever called.None; cross-provider collision test asserts the bare id routes to neither.test_app.pypooling/classify fixtures moved to namespaced ids (they used single-segment"m").🤖 Generated with Claude Code