[Test] Add E2E coverage for the language signal - #3507
Open
yu3zhang1 wants to merge 3 commits into
Open
Conversation
yu3zhang1
requested review from
AayushSaini101,
FAUST-BENCHOU,
WUKUNTAI-0211,
Xunzhuo,
drivebyer,
ramkrishs,
rootfs,
shraderdm,
theohsiung and
wilsonwu
as code owners
September 5, 2026 04:26
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
Author
|
E2E (envoy-ai-gateway) runs successfully: |
yu3zhang1
force-pushed
the
test/language-signal-e2e-3178
branch
from
September 5, 2026 17:38
5d810e7 to
ae64a0e
Compare
drivebyer
reviewed
Sep 7, 2026
drivebyer
left a comment
Collaborator
There was a problem hiding this comment.
Non-blocker: the load/result/print scaffolding here mirrors event_routing.go almost verbatim. Could you extract a shared helper in this PR and reuse it in both cases?
* Add an `es`(Spanish) language rule and a `spanish_language` decision to the ai-gateway profile. * Add a new language-routing test case containing two positive cases (distinct Spanish-language queries) and one negative case (an English, non-Spanish query). * Register language-routing in BaselineRouterContract, the signal-routing contract test, and the envoy-ai-gateway CI test list. Second bounded slice of the signal/projection E2E coverage requested in vllm-project#3178 (Evaluation & Quality workgroup); remaining signals will land as separate follow-up slices. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: pikachu <4401183+yu3zhang1@users.noreply.github.com>
yu3zhang1
force-pushed
the
test/language-signal-e2e-3178
branch
from
September 7, 2026 15:47
5988a27 to
f079dd4
Compare
Addresses review feedback from @drivebyer on vllm-project#3507: event_routing.go and language_routing.go duplicated the same load/loop/print scaffolding almost verbatim. * Extract it into `signal_routing_helpers.go` and reuse it from both, * Unifying the testdata field name to `expected_matched_signal`. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: pikachu <4401183+yu3zhang1@users.noreply.github.com>
CI's Full pre-commit checks job failed: gci wants a separate import group for github.com/vllm-project/semantic-router-prefixed imports, after the standard-library and third-party groups. Reorder imports in event_routing.go, language_routing.go, signal_routing_helpers.go, and signal_routing_contract_test.go to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: pikachu <4401183+yu3zhang1@users.noreply.github.com>
Contributor
Author
Done. Move the shared code code to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related #3178
Purpose
Add e2e test for the language routing signal (config.SignalTypeLanguage).
It currently has only Go unit tests for its classifier logic
(
language_classifier_test.go) and no end-to-end behavioral coveragethrough a deployed request path, as required by #3178.
This PR adds:
es(Spanish) language rule and aspanish_languagedecision(priority 25) to
e2e/profiles/ai-gateway/values.yaml.e2e/testcases/language_routing.go, following the existingpkgtestcases.Register(...)pattern used byevent_routing.go,asserting the
x-vsr-selected-decisionandx-vsr-matched-languageresponse headers.
e2e/testcases/testdata/language_routing_cases.jsonwith two positivecases (distinct Spanish-language queries) and one negative case (an
English, non-Spanish query).
language-routingregistered inBaselineRouterContract, a new row insignal_routing_contract_test.go, and theENVOY_AI_GATEWAY_CI_TESTSlist in
.github/workflows/integration-test-k8s.yml— all threeregistration points needed for the test to actually run in CI, added
up front based on review feedback from [Test] Add E2E coverage for the event signal #3399.
Owning Workgroup: Evaluation & Quality (
wg/evaluation-quality).Test Plan
gofmt -l e2e/testcases/language_routing.gogo vet ./...(run frome2e/)go build ./...(run frome2e/)golangci-lint run ./testcases/... ./pkg/testmatrix/...(run frome2e/)go test ./testcases/... -run TestProfilesSelectSignalRoutingContractsmake e2e-test-specific E2E_TESTS="language-routing"Test Result
issues in unrelated files under
e2e/testcases/were left untouched).TestProfilesSelectSignalRoutingContracts: pass, confirms theenvoy-ai-gatewayprofile selectslanguage-routing.make e2e-test-specific: not run against a live cluster locally(same local Kubernetes/network constraints noted on [Test] Add E2E coverage for the event signal #3399). Deferring
final e2e confirmation to CI on this PR.
values.yaml: the newspanish_languagedecision (priority 25) sits betweenthinking_decision(20) and
urgent_request(30), and its Spanish-language test queriesdon't contain any of the other decisions' trigger keywords, so the new
test cases can't be shadowed by an unrelated decision.
AI assistance (Claude Code) was used to draft this change, under my review;
I read and understand the diff and take responsibility for it.
🤖 Generated with Claude Code