[Router] export Router Learning experience as versioned snapshots - #2712
Open
EuclidStellar wants to merge 4 commits into
Open
[Router] export Router Learning experience as versioned snapshots#2712EuclidStellar wants to merge 4 commits into
EuclidStellar wants to merge 4 commits into
Conversation
vllm-project#2240 needs a durable, versioned view of what Router Learning has observed per decision/tier/model. Define that as its own exported type in routerruntime (parallel to RouterOutcome/OutcomeRuntime) rather than reusing extproc's private experience struct, so the runtime's internal representation can change without breaking whatever reads the exported snapshot later (CLI, API, materializer). Config/recipe and data-window identity are deliberately left out here pending the shared-identity-type question raised on the issue; this only versions the fields that already have real identity today. Signed-off-by: euclidstellar <euclidstellar@gmail.com>
Key the in-process experience map by a typed struct instead of a formatted string, so it can be read back without parsing. Add ExperienceSnapshots() to convert it to the versioned contract from the previous commit. Read-only: nothing consumes these snapshots yet, so routing behavior is unchanged. Update the apiserver test fake to satisfy the wider LearningRuntime interface. Signed-off-by: euclidstellar <euclidstellar@gmail.com>
Covers the exact/tier/global fallback triple an outcome writes and that the exported snapshot carries the right identity, evidence, and schema version. Signed-off-by: euclidstellar <euclidstellar@gmail.com>
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned when their GitHub accounts are assignable in this repository: 📁
|
Contributor
✅ Supply Chain Security Report — All Clear
Scanned at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #2240
Purpose
routerLearningModelExperience, keyed by decision/tier/model) only ever lives in-process and is only ever consumed by the live scorer — there's no way to read it back as data.RouterWhat this does
routerruntime.RouterExperienceSnapshot(new, exported): a versioned, stable-contract copy of one decision/tier/model experience entry — separate from extproc's internal struct so one can change without breaking the other. IncludesSchemaVersion, identity (Decision/Tier/Model), the existing evidence fields, a derivedSampleCount, and aSourcefield ("runtime"today, so runtime vs. future imported evidence stays distinguishable per scope item 6 — the value it needs later, not the import path itself).routerruntime.ExperienceRuntime(new interface, folded intoLearningRuntime):ExperienceSnapshots() []RouterExperienceSnapshot.extproc: the experience map is now keyed by a typedrouterLearningExperienceKeystruct instead of a formatted"decision|tier|model"string — needed to read entries back without writing a string parser, and incidentally removes a stringly-typed key that had no real reason to be a string.ExperienceSnapshots()implements the new interface by converting the map to the versioned type.Explicit non-goals for this PR
Test Plan
go build/go vetclean onrouterruntime,extproc,apiserver.make agent-lintgate (which also caught and required fixing a test fake inpkg/apiserverthat implemented the now-widerLearningRuntimeinterface).TestRouterLearningRuntimeExperienceSnapshots, covering the existing exact/tier/global fallback triple one outcome writes, asserting the exported snapshot's identity, evidence, sample count, schema version, and source.Test Result
make agent-lintgate all pass.go testexecution forextproc/routerruntimeisn't possible in this sandbox (pre-existing: linking their test binaries requires a compiledcandle-bindingRust library not present here — same limitation noted on [Router] persist HaluGate NLI span detail in Router Replay #2609). Traced the new test's execution by hand against the actual code path (UpdateOutcome→recordModelExperience's 3-way fallback write →ExperienceSnapshots) and confirmed the assertions hold; flagging for CI/a maintainer to confirm the actual run.Semantic Router PR Checklist
[Router],[CLI],[Dashboard],[Operator],[Fleet-Sim],[Bindings],[Training],[E2E],[Docs], or[CI/Build]git commit -s