[Bug] Route the category-remote-backend E2E through a real gateway route - #3544
Open
theohsiung wants to merge 4 commits into
Open
[Bug] Route the category-remote-backend E2E through a real gateway route#3544theohsiung wants to merge 4 commits into
theohsiung wants to merge 4 commits into
Conversation
The profile has never passed. Its job failed on the PR that introduced it and has kept failing, so it reported coverage of the remote category backend that it did not provide. The router was never at fault: it starts cleanly and logs a correctly selected backend, and the request simply never reaches it. The profile named its model base-model, while the shared AIGatewayRoute matches x-ai-eg-model exactly against math-, science-, social-, humanities-, law- and general-expert. A model outside that list has no route, so the gateway answers 404 - and because the router looks healthy, the failure reads like a router bug. Verified on a kind cluster: the profile now passes, where the same profile 404s without this change. Signed-off-by: theohsiung <theobear870924@gmail.com>
theohsiung
requested review from
AayushSaini101,
FAUST-BENCHOU,
WUKUNTAI-0211,
drivebyer,
ramkrishs,
shraderdm and
wilsonwu
as code owners
September 7, 2026 09:49
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
github-actions
Bot
added
the
pr/blocked
theohsiung
added
the
wg/router-models-inference-runtime
The profile had two independent faults, and a roomier machine only shows the second. At 2Gi the router is OOMKilled on a CI runner, restarts, never becomes ready, and the helm wait burns its full 60m timeout - which is what the 1h23m job actually was, not a slow test. It still loads the local mmbert embedding model the response cache is configured with, and taking the classifier off-box does not avoid that: the embedding models are gated separately from the classifier backend. A local run survives 2Gi and fails later at the gateway with a 404 instead, which is how one broken profile looked like two different bugs. Signed-off-by: theohsiung <theobear870924@gmail.com>
…-route Signed-off-by: theohsiung <theobear870924@gmail.com> # Conflicts: # e2e/profiles/category-remote-backend/values.yaml
github-actions
Bot
added
pr/needs-rebase
Contributor
|
Queued — the merge queue status continues in this comment ↓. |
Contributor
Merge Queue Status
This pull request spent 7 seconds in the queue, with no time running CI. ReasonThe pull request can't be updated
HintYou should update or rebase your pull request manually. If you do, this pull request will automatically be requeued once the queue conditions match again. Tick the box to put this pull request back in the merge queue (same as
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
category-remote-backendhas never passed — its job failed on the PR thatintroduced it (#3138) and has kept failing, so it reported coverage of the
remote category backend that it did not provide.
The router was never at fault. It starts cleanly and logs a correctly selected
backend; the request never reaches it. The profile named its model
base-model, while the sharedAIGatewayRoutematchesx-ai-eg-modelexactlyagainst
math-,science-,social-,humanities-,law-andgeneral-expert. Anything outside that list has no route, so the gatewayanswers 404 — and because the router looks healthy, it reads like a router bug.
One line of substance: the model becomes
general-expert, whiche2e/profiles/ai-gateway/values.yamlalready uses for the same reason. Therest is a comment so the next profile on this stack does not repeat it.
Validation
Run on a kind cluster, before and after:
Closes #3543