Conversation
Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai>
✅ Deploy Preview for vllm-semantic-router ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
github-actions
Bot
assigned
abdallahsamabd,
asaadbalum,
haowu1234,
JaredforReal,
liavweiss,
noalimoy,
rootfs,
samzong,
szedan-rh and
Xunzhuo
Contributor
👥 vLLM Semantic Team NotificationThe following members have been identified for the changed files in this PR and have been automatically assigned: 📁
|
Contributor
✅ Supply Chain Security Report — All Clear
Scanned at |
Contributor
Performance Benchmark ResultsComponent benchmarks completed successfully. Summary
DetailsSee attached benchmark artifacts for detailed results and profiles. Performance testing powered by vLLM Semantic Router |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new “session-aware routing” public surface by introducing a session signal family plus a session_aware selection algorithm, while retiring the legacy routing.session_states / SESSION_STATE DSL surface. This fits into the router’s routing-surface + selection-algorithm architecture and extends replay/telemetry to persist multi-turn metadata needed for lookups and debugging.
Changes:
- Introduces
routing.signals.session+ runtime injection of session facts into signal evaluation, projections, and decision conditions. - Adds
session_awareselection algorithm (config + selector implementation) and wires it through extproc selection context + lookup tables. - Removes legacy
SESSION_STATEDSL/config surface and updates docs/recipes/manifests accordingly.
Reviewed changes
Copilot reviewed 95 out of 95 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/tutorials/signal/overview.md | Adds session to the signal tutorial index. |
| website/docs/tutorials/signal/heuristic/session.md | New tutorial page describing the session signal family. |
| website/docs/tutorials/algorithm/selection/session-aware.md | New tutorial page for session_aware selection. |
| website/docs/tutorials/algorithm/overview.md | Links the new session-aware algorithm tutorial. |
| tools/agent/repo-manifest.yaml | Registers the new session-aware routing convergence-loop plan. |
| docs/agent/plans/pl-0030-session-aware-routing-convergence-loop.md | New execution plan documenting the session-aware routing workstream and removal of legacy session_state surface. |
| docs/agent/plans/README.md | Adds the new plan to the plans index. |
| src/vllm-sr/cli/config_contract.py | Adds session to the CLI signal-family contract inventory. |
| src/vllm-sr/cli/models.py | Adds signals.session field (but currently missing its model type). |
| src/vllm-sr/cli/algorithms.py | Adds algorithm.session_aware field (but currently missing its model type). |
| src/vllm-sr/cli/validator.py | Adds session_aware to algorithm type lists and attempts to validate it (but validator function is missing). |
| src/semantic-router/pkg/config/config.go | Adds SignalTypeSession and removes SessionStates from routing config surface. |
| src/semantic-router/pkg/config/signal_config.go | Adds SessionRules to signals config struct. |
| src/semantic-router/pkg/config/session_signal_config.go | Introduces session fact constants + SessionRule contract helpers. |
| src/semantic-router/pkg/config/session_selection_config.go | Adds config schema for session_aware algorithm parameters. |
| src/semantic-router/pkg/config/selection_config.go | Adds global model-selection session_aware block. |
| src/semantic-router/pkg/config/decision_config.go | Adds algorithm.session_aware decision block. |
| src/semantic-router/pkg/config/routing_surface_catalog.go | Registers session signal type and session_aware algorithm as supported. |
| src/semantic-router/pkg/config/validator.go | Hooks session contract validation into config validation chain. |
| src/semantic-router/pkg/config/validator_session.go | Adds validation for session signals + session-aware algorithm config. |
| src/semantic-router/pkg/config/validator_decision.go | Validates algorithm.type=session_aware and its config block. |
| src/semantic-router/pkg/config/validator_projection.go | Allows projection inputs of type session and validates declared session signals. |
| src/semantic-router/pkg/config/validator_projection_test.go | Adds test that session signals can be used as projection inputs. |
| src/semantic-router/pkg/config/reference_config_public_surface_test.go | Removes session_state coverage checks and adds session signal reference coverage. |
| src/semantic-router/pkg/config/reference_config_routing_surface_test.go | Adds session signal key mapping and session_aware algorithm coverage assertion. |
| src/semantic-router/pkg/config/canonical_config.go | Adds canonical session signals; removes canonical session_states mapping. |
| src/semantic-router/pkg/config/canonical_export.go | Exports canonical session signals; stops exporting session_states. |
| src/semantic-router/pkg/config/fragment_catalog_test.go | Registers the session-aware algorithm fragment path. |
| src/semantic-router/pkg/selection/selector.go | Adds MethodSessionAware and extends SelectionContext with session metadata + lookup priors. |
| src/semantic-router/pkg/selection/session_aware.go | New SessionAwareSelector implementation. |
| src/semantic-router/pkg/selection/tier_declarations.go | Marks SessionAwareSelector as supported tier. |
| src/semantic-router/pkg/selection/factory.go | Wires session_aware into selector factory + registry. |
| src/semantic-router/pkg/selection/lookuptable/builder.go | Uses persisted SessionID for replay grouping when available (fallback to heuristic for older data). |
| src/semantic-router/pkg/extproc/router.go | Injects lookup table handle into router struct. |
| src/semantic-router/pkg/extproc/router_build.go | Propagates lookup table through router component construction. |
| src/semantic-router/pkg/extproc/router_selection.go | Wires session_aware method/config into model-selection config and registry creation. |
| src/semantic-router/pkg/extproc/router_selection_context.go | Builds SelectionContext including session facts and replay-derived priors. |
| src/semantic-router/pkg/extproc/request_context.go | Adds fields for session identity and matched session signals. |
| src/semantic-router/pkg/extproc/req_filter_classification_session.go | Computes runtime session signal values and matched session rules. |
| src/semantic-router/pkg/extproc/req_filter_classification_signal.go | Copies matched session signals into request context and “all matched rules” list. |
| src/semantic-router/pkg/extproc/req_filter_classification_runtime.go | Adds session_aware to algorithm→method map; logs matched session rules; calls runtime session signal injection. |
| src/semantic-router/pkg/extproc/req_filter_classification.go | Refactors selection-context creation into a helper that includes session facts. |
| src/semantic-router/pkg/extproc/recorder.go | Records session metadata + session signals into replay records. |
| src/semantic-router/pkg/extproc/recorder_test.go | Extends extproc recorder tests for session metadata + session signals. |
| src/semantic-router/pkg/routerreplay/store/store.go | Extends replay record schema to include session metadata + session signals. |
| src/semantic-router/pkg/routerreplay/store/postgres.go | Adds Postgres columns + index to persist session-aware replay metadata. |
| src/semantic-router/pkg/routerreplay/store/postgres_record_row.go | Updates Postgres row scan/insert to include new session fields. |
| src/semantic-router/pkg/routerreplay/recorder.go | Logs replay fields including session metadata + session signals. |
| src/semantic-router/pkg/routerreplay/recorder_test.go | Extends routerreplay logging tests for new fields. |
| src/semantic-router/pkg/decision/engine.go | Adds session rule matches to the decision engine match bundle. |
| src/semantic-router/pkg/classification/classifier_signal_eval.go | Threads matched session signals into debug logs and decision-engine inputs. |
| src/semantic-router/pkg/dsl/ast.go | Removes SESSION_STATE from the DSL AST. |
| src/semantic-router/pkg/dsl/parser.go | Removes SESSION_STATE parsing support. |
| src/semantic-router/pkg/dsl/compiler.go | Removes session_state compilation; adds compilation of SIGNAL session + ALGORITHM session_aware. |
| src/semantic-router/pkg/dsl/decompiler.go | Removes session_state decompilation; adds decompile support for session signals + session-aware algorithm fields. |
| src/semantic-router/pkg/dsl/routing_contract.go | Removes SESSION_STATES section from decompile output; adds session signals to operational signals. |
| src/semantic-router/pkg/dsl/validator_conflicts.go | Removes legacy SESSION_STATE conflict checks. |
| src/semantic-router/pkg/dsl/maintained_asset_roundtrip_test.go | Removes maintained recipe sync test for session-state assets. |
| src/semantic-router/pkg/dsl/dsl_test.go | Removes SESSION_STATE tests; adds session_aware algorithm compile/decompile tests. |
| deploy/recipes/session-state.yaml | Removes legacy session-state recipe YAML. |
| deploy/recipes/session-state.dsl | Removes legacy session-state recipe DSL. |
| config/signal/session/runtime-facts.yaml | Adds shipped session-signal fragments for runtime facts. |
| config/algorithm/selection/session-aware.yaml | Adds shipped algorithm fragment for session-aware selection. |
| config/config.yaml | Removes routing.session_states; adds a session-aware decision using session signal + session_aware algorithm. |
| config/README.md | Documents session_aware as an available selection algorithm fragment. |
| src/semantic-router/pkg/k8s/testdata/output/01-basic.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/02-keyword-only.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/03-embedding-only.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/04-domain-only.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/05-keyword-embedding.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/06-keyword-domain.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/07-domain-embedding.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/08-keyword-embedding-domain.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/09-keyword-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/10-embedding-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/11-domain-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/12-keyword-embedding-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/13-keyword-domain-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/14-domain-embedding-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/15-keyword-embedding-domain-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| src/semantic-router/pkg/k8s/testdata/output/16-keyword-embedding-domain-no-plugin.yaml | Updates expected rendered config (startup_status block addition). |
| dashboard/backend/handlers/topology.go | Extends matched-signal payload to include session signals. |
| dashboard/backend/handlers/topology_response.go | Adds session signals to topology response mapping. |
| dashboard/frontend/src/types/config.ts | Adds SessionSignal type and expands decision condition types (but signals shape is incomplete). |
| dashboard/frontend/src/pages/topology/utils/topologyParser.ts | Parses session signals and session_aware algorithm config into topology graph nodes. |
| dashboard/frontend/src/pages/topology/types.ts | Adds session signal + session-aware algorithm type definitions. |
| dashboard/frontend/src/pages/topology/constants.ts | Adds icon/color for session signal and color for session_aware algorithm. |
| dashboard/frontend/src/pages/topology/components/CustomNodes/AlgorithmNode.tsx | Renders session_aware algorithm summary in the topology UI. |
| dashboard/frontend/src/pages/configPageSupport.ts | Extends config-page support types to include session signal type and model-selection session_aware config. |
| dashboard/frontend/src/pages/configPageRouterDefaultsSupport.ts | Adds session_aware to model selection defaults editor and persistence. |
| dashboard/frontend/src/pages/DashboardPage.tsx | Adds session and kb colors to flow diagram legend rendering. |
| dashboard/frontend/src/pages/ConfigPageSignalsSection.tsx | Adds UI for listing/editing/creating session signals. |
| dashboard/frontend/src/pages/ConfigPageDecisionsSection.tsx | Allows decisions to reference session and kb signal names. |
| dashboard/frontend/src/pages/ConfigPage.tsx | Adds deletion support for session signals. |
Comment on lines
346
to
350
| structure: Optional[List[StructureRule]] = [] | ||
| complexity: Optional[List[ComplexityRule]] = [] | ||
| modality: Optional[List[ModalityRule]] = [] | ||
| session: Optional[List[SessionRule]] = [] | ||
| role_bindings: Optional[List[RoleBindingRule]] = [] |
There was a problem hiding this comment.
Signals.session is typed as List[SessionRule], but SessionRule is not defined or imported in this file. Because this file does not use from __future__ import annotations, this will fail at import time. Add a SessionRule Pydantic model (matching the router contract: name, fact, predicate, intent_or_domain, previous_model, candidate_model, description) or import it from the correct module.
Comment on lines
347
to
353
| # Selection algorithm configurations (from PR #1089, #1104) | ||
| elo: EloSelectionConfig | None = None | ||
| router_dc: RouterDCSelectionConfig | None = None | ||
| automix: AutoMixSelectionConfig | None = None | ||
| hybrid: HybridSelectionConfig | None = None | ||
| session_aware: SessionAwareSelectionConfig | None = None | ||
|
|
There was a problem hiding this comment.
AlgorithmConfig.session_aware references SessionAwareSelectionConfig, but that type is not defined or imported in this module. This will raise a NameError when evaluating annotations. Add a SessionAwareSelectionConfig Pydantic model (and any validation constraints) or import it from the module where it is defined.
Comment on lines
211
to
235
| export interface Signals { | ||
| keywords?: KeywordSignal[] | ||
| embeddings?: EmbeddingSignal[] | ||
| domains?: DomainSignal[] | ||
| fact_check?: FactCheckSignal[] | ||
| user_feedbacks?: UserFeedbackSignal[] | ||
| reasks?: ReaskSignal[] | ||
| preferences?: PreferenceSignal[] | ||
| language?: LanguageSignal[] | ||
| context?: ContextSignal[] | ||
| structure?: StructureSignal[] | ||
| complexity?: ComplexitySignal[] | ||
| modality?: ModalitySignal[] | ||
| role_bindings?: RoleBindingSignal[] | ||
| jailbreak?: JailbreakSignal[] | ||
| pii?: PIISignal[] | ||
| } | ||
|
|
||
| // ============================================================================= | ||
| // DECISIONS - Routing logic | ||
| // ============================================================================= | ||
|
|
||
|
|
||
| export type DecisionConditionType = 'keyword' | 'domain' | 'preference' | 'user_feedback' | 'reask' | 'embedding' | 'fact_check' | 'language' | 'context' | 'structure' | 'complexity' | 'modality' | 'authz' | 'jailbreak' | 'pii' | 'projection' | ||
| export type DecisionConditionType = 'keyword' | 'domain' | 'preference' | 'user_feedback' | 'reask' | 'embedding' | 'fact_check' | 'language' | 'context' | 'structure' | 'complexity' | 'modality' | 'session' | 'authz' | 'jailbreak' | 'pii' | 'projection' | 'kb' | ||
| export interface DecisionCondition { |
There was a problem hiding this comment.
SessionSignal is defined and DecisionConditionType now includes session/kb, but the Signals interface does not include session?: SessionSignal[] (and also lacks a kb signal type). This will make config.signals.session / config.signals.kb usages in the UI type-unsafe and can break TypeScript builds. Add the missing properties (and corresponding signal interfaces) to Signals.
Comment on lines
+21
to
+29
| if !IsSupportedSessionFact(rule.Fact) { | ||
| return fmt.Errorf("fact %q is unsupported", rule.Fact) | ||
| } | ||
| if rule.Predicate == nil { | ||
| return fmt.Errorf("predicate is required") | ||
| } | ||
| if err := validateNumericPredicate(rule.Predicate); err != nil { | ||
| return fmt.Errorf("predicate: %w", err) | ||
| } |
There was a problem hiding this comment.
validateSessionRule currently rejects session signals without a predicate (predicate is required), but the new docs/UI treat predicate as optional. This also differs from other numeric-predicate signals (e.g., structure) where a nil predicate is allowed. Either (a) allow predicate to be omitted (and only validate predicate when provided), or (b) update the documented/public contract + UI to require predicate for session signals.
| { label: 'Candidate Model', value: signal.rawData.candidate_model || 'Any' }, | ||
| { | ||
| label: 'Predicate', | ||
| value: signal.rawData.predicate ? JSON.stringify(signal.rawData.predicate, null, 2) : 'Always match', |
There was a problem hiding this comment.
This UI treats a missing session predicate as "Always match" and allows saving predicate: undefined, but the router-side config validator currently rejects session signals without a predicate. Align the frontend with the backend contract (either require a predicate here, or change backend validation/runtime semantics to allow omitted predicates).
Suggested change
| value: signal.rawData.predicate ? JSON.stringify(signal.rawData.predicate, null, 2) : 'Always match', | |
| value: signal.rawData.predicate | |
| ? JSON.stringify(signal.rawData.predicate, null, 2) | |
| : 'Missing predicate (invalid configuration)', |
Comment on lines
+214
to
+223
| intelligentRouting := cfg.IntelligentRouting | ||
| sessionCfg := intelligentRouting.ModelSelection.SessionAware | ||
| result := &selection.SessionAwareConfig{ | ||
| FallbackMethod: sessionCfg.FallbackMethod, | ||
| MinTurnsBeforeSwitch: sessionCfg.MinTurnsBeforeSwitch, | ||
| StayBias: sessionCfg.StayBias, | ||
| QualityGapMultiplier: sessionCfg.QualityGapMultiplier, | ||
| HandoffPenaltyWeight: sessionCfg.HandoffPenaltyWeight, | ||
| RemainingTurnWeight: sessionCfg.RemainingTurnWeight, | ||
| } |
There was a problem hiding this comment.
buildSessionAwareSelectionConfig always constructs a non-nil *selection.SessionAwareConfig from intelligentRouting.ModelSelection.SessionAware. Because SessionAware is a value struct in config, an omitted block becomes all-zero and this will override the selector defaults (e.g., stay_bias becomes 0 instead of 0.25). Consider starting from selection.DefaultSessionAwareConfig() and only overriding fields that are explicitly set in config, or returning nil when both global+decision configs are effectively empty so the factory can apply defaults.
Comment on lines
+62
to
+70
| | Parameter | Required | Description | | ||
| |-----------|----------|-------------| | ||
| | `name` | yes | Signal name referenced from decisions and projections | | ||
| | `fact` | yes | Runtime-derived fact name injected by the router | | ||
| | `predicate` | no | Numeric threshold predicate over the fact value | | ||
| | `intent_or_domain` | no | Optional domain/task-family guard | | ||
| | `previous_model` | no | Only match when the previous turn used this model | | ||
| | `candidate_model` | no | Only match when evaluating this candidate model | | ||
| | `description` | no | Human-readable explanation | |
There was a problem hiding this comment.
The parameter table marks predicate as not required, but the current router config validator rejects session signals without a predicate. Please align this doc with the enforced contract (either document predicate as required, or update backend validation/runtime to allow omitted predicates).
Comment on lines
568
to
571
| errors.extend(validate_algorithm_one_of(config)) | ||
| errors.extend(validate_latency_aware_algorithm_config(config)) | ||
| errors.extend(validate_session_aware_algorithm_config(config)) | ||
|
|
There was a problem hiding this comment.
validate_user_config() calls validate_session_aware_algorithm_config(config), but that function is not defined anywhere in this module (and no import provides it). This will raise a NameError at runtime and break config validation. Define validate_session_aware_algorithm_config (mirroring the existing validate_latency_aware_algorithm_config pattern) or remove the call until the validator is implemented.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PLEASE FILL IN THE PR DESCRIPTION BELOW AND CONFIRM THE CHECKLIST ITEMS.
Closes #xxxx
Purpose
Router/CLI/Dashboard/Operator/Fleet-Sim/Bindings/Training/E2E/Docs/CI/BuildTest Plan
Test Result
Semantic Router PR Checklist
[Router],[CLI],[Dashboard],[Operator],[Fleet-Sim],[Bindings],[Training],[E2E],[Docs], or[CI/Build]git commit -sSee CONTRIBUTING.md for the full contributor workflow and commit guidance.