Skip to content

[Router] export Router Learning experience as versioned snapshots - #2712

Open
EuclidStellar wants to merge 4 commits into
vllm-project:mainfrom
EuclidStellar:router-learning-experience-snapshot
Open

[Router] export Router Learning experience as versioned snapshots#2712
EuclidStellar wants to merge 4 commits into
vllm-project:mainfrom
EuclidStellar:router-learning-experience-snapshot

Conversation

@EuclidStellar

@EuclidStellar EuclidStellar commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

closes #2240

Purpose

  • Build Router Learning experience materialization #2240 needs a versioned, exportable view of Router Learning's experience state. Today that state (routerLearningModelExperience, keyed by decision/tier/model) only ever lives in-process and is only ever consumed by the live scorer — there's no way to read it back as data.
  • This is a deliberately narrow first slice, matching the issue's own "define export first" ordering (scope item 4): a versioned snapshot type + a read-only export of current state. No async materializer, no storage backend, no import/reload path, no config/data-window identity yet — those are separate, larger decisions I've flagged as open questions on the issue and haven't heard back on.
  • Given Harden the Router Learning runtime contract #2239 (the runtime contract hardening this issue depends on) is also unassigned with no response yet, I scoped this to something that doesn't touch the pipeline, strategy registry, or protection/candidate semantics Harden the Router Learning runtime contract #2239 is about — it's a read-only export of state that already exists, so it shouldn't need to wait on that contract being formally written up.
  • Module: Router

What this does

  • routerruntime.RouterExperienceSnapshot (new, exported): a versioned, stable-contract copy of one decision/tier/model experience entry — separate from extproc's internal struct so one can change without breaking the other. Includes SchemaVersion, identity (Decision/Tier/Model), the existing evidence fields, a derived SampleCount, and a Source field ("runtime" today, so runtime vs. future imported evidence stays distinguishable per scope item 6 — the value it needs later, not the import path itself).
  • routerruntime.ExperienceRuntime (new interface, folded into LearningRuntime): ExperienceSnapshots() []RouterExperienceSnapshot.
  • extproc: the experience map is now keyed by a typed routerLearningExperienceKey struct instead of a formatted "decision|tier|model" string — needed to read entries back without writing a string parser, and incidentally removes a stringly-typed key that had no real reason to be a string. ExperienceSnapshots() implements the new interface by converting the map to the versioned type.
  • Nothing consumes these snapshots yet. Read-only, zero behavior change to routing.

Explicit non-goals for this PR

  • No async materializer, no storage/persistence, no reload/import, no multi-replica semantics (that's Define distributed Router Learning state semantics #2243's).
  • No config/recipe/data-window identity fields — open question on the issue, not resolved here.
  • No uncertainty/statistical confidence metric beyond the raw sample count.

Test Plan

  • go build / go vet clean on routerruntime, extproc, apiserver.
  • Full local make agent-lint gate (which also caught and required fixing a test fake in pkg/apiserver that implemented the now-wider LearningRuntime interface).
  • Added TestRouterLearningRuntimeExperienceSnapshots, covering the existing exact/tier/global fallback triple one outcome writes, asserting the exported snapshot's identity, evidence, sample count, schema version, and source.

Test Result

  • Build/vet/golangci-lint (0 issues) and the full make agent-lint gate all pass.
  • Full go test execution for extproc/routerruntime isn't possible in this sandbox (pre-existing: linking their test binaries requires a compiled candle-binding Rust library not present here — same limitation noted on [Router] persist HaluGate NLI span detail in Router Replay #2609). Traced the new test's execution by hand against the actual code path (UpdateOutcomerecordModelExperience's 3-way fallback write → ExperienceSnapshots) and confirmed the assertions hold; flagging for CI/a maintainer to confirm the actual run.

Semantic Router PR Checklist
  • PR title uses module-aligned prefixes such as [Router], [CLI], [Dashboard], [Operator], [Fleet-Sim], [Bindings], [Training], [E2E], [Docs], or [CI/Build]
  • If the PR spans multiple modules, the title includes all relevant prefixes
  • Commits in this PR are signed off with git commit -s
  • The Purpose, Test Plan, and Test Result sections reflect the actual scope, commands, and blockers for this change

vllm-project#2240 needs a durable, versioned view of what Router Learning has
observed per decision/tier/model. Define that as its own exported
type in routerruntime (parallel to RouterOutcome/OutcomeRuntime)
rather than reusing extproc's private experience struct, so the
runtime's internal representation can change without breaking
whatever reads the exported snapshot later (CLI, API, materializer).

Config/recipe and data-window identity are deliberately left out here
pending the shared-identity-type question raised on the issue; this
only versions the fields that already have real identity today.

Signed-off-by: euclidstellar <euclidstellar@gmail.com>
Key the in-process experience map by a typed struct instead of a
formatted string, so it can be read back without parsing. Add
ExperienceSnapshots() to convert it to the versioned contract from
the previous commit. Read-only: nothing consumes these snapshots yet,
so routing behavior is unchanged.

Update the apiserver test fake to satisfy the wider LearningRuntime
interface.

Signed-off-by: euclidstellar <euclidstellar@gmail.com>
Covers the exact/tier/global fallback triple an outcome writes and
that the exported snapshot carries the right identity, evidence, and
schema version.

Signed-off-by: euclidstellar <euclidstellar@gmail.com>
@netlify

netlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit f527928
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/6a6897385d396b000885768e
😎 Deploy Preview https://deploy-preview-2712--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions

Copy link
Copy Markdown
Contributor

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned when their GitHub accounts are assignable in this repository:

📁 src/semantic-router

Owners: @FAUST-BENCHOU, @shraderdm, @drivebyer, @ramkrishs, @WUKUNTAI-0211, @AayushSaini101, @siloteemu
Files changed:

  • src/semantic-router/pkg/apiserver/route_router_outcomes_test.go
  • src/semantic-router/pkg/extproc/router_learning_experience_snapshot.go
  • src/semantic-router/pkg/extproc/router_learning_runtime.go
  • src/semantic-router/pkg/extproc/router_learning_runtime_test.go
  • src/semantic-router/pkg/routerruntime/learning_experience.go
  • src/semantic-router/pkg/routerruntime/registry.go

vLLM Semantic Router

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Supply Chain Security Report — All Clear

Scanner Status Findings
AST Codebase Scan (Py, Go, JS/TS, Rust) 31 finding(s) — MEDIUM: 24 · LOW: 7
AST PR Diff Scan No issues detected
Regex Fallback Scan No issues detected

Scanned at 2026-07-28T12:57:56.832Z · View full workflow logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build Router Learning experience materialization

7 participants