[Feature] Add typed success estimates and immutable evidence snapshots - #3518
[Feature] Add typed success estimates and immutable evidence snapshots#3518sujal111 wants to merge 1 commit into
Conversation
Freeze in-memory Router Learning experience at request start and report conservative typed estimates so later observe-only selection can rank on calibrated P(success) without treating QualitySeed as a probability. Signed-off-by: SUJAL MAITI <sujal@SUJALs-MacBook-Pro.local> Co-authored-by: Cursor <cursoragent@cursor.com>
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Xunzhuo
left a comment
There was a problem hiding this comment.
One core part of #3480 is not wired into the observe path yet. attachSuccessEstimateObserveDiagnostics calls estimateCandidateSuccess with no stale horizon, so StaleAfter stays zero and stale evidence can never produce the stale result. The same path uses resolveExperience, which simply picks the first populated scope; mergeScopedExperience is never called, so conflicting scopes are silently accepted and the conflict result is unreachable. The direct helper tests do not exercise these production paths. Please wire the decision/global observe controls through the real lookup path and add observe-path coverage showing stale and conflicting evidence fail conservatively.
Add typed success estimates and an immutable in-memory evidence snapshot under Router Learning (#3480).
Observe mode records the estimates in Router Replay. The selected model does not change.
routing_samplingstays the default — this PR does not addsuccess_constrainedor apply mode.Closes #3480
Related #3412
Related #3494
Purpose
First implementation PR after the proposal.
pkg/extprocnow freezes the in-memory experience at request start and reports a typed estimate per candidate (calibrated/insufficient_evidence/unsupported/stale/conflict). Seed-onlyQualitySeedand classifier/similarity scores are not treated asP(success).wg/mom-routing
Next PRs
success_constrained(feasibility filter, lifecycle-cost ranking, fallback, decision-local success config). Still observe-only.Test Plan
go test ./pkg/extproc/ -run 'SuccessEstimate|TestRouterLearning|TestBuildReplayRoutingRecordCapturesLearning'Test Result
Targeted extproc run passed locally. Full
make test-semantic-routeris on CI — I could not rebuild the candle bindings here.