Add routing_method experiment variable for A/B/C hydra experiment#4947
Open
aashna wants to merge 1 commit intomicrosoft:mainfrom
Open
Add routing_method experiment variable for A/B/C hydra experiment#4947aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna wants to merge 1 commit intomicrosoft:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds richer Auto Mode model-routing behavior (including HYDRA routing metadata) and wires post-summarization re-routing so the system can switch models at a compaction boundary.
Changes:
- Adds per-conversation routing (route once on first turn, reuse thereafter) plus related cache/telemetry in
AutomodeService. - Extends router decision shape to include HYDRA/fallback metadata and introduces an experiment/config key for routing method selection.
- Wires an optional post-summarization callback through the agent prompt/summarizer flow to allow re-routing after a summary is produced.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/platform/endpoint/node/test/automodeService.spec.ts | Adds unit tests for per-conversation routing vs per-turn routing behavior. |
| src/platform/endpoint/node/routerDecisionFetcher.ts | Extends router response typing/logging and adds an (intended) routing method parameter. |
| src/platform/endpoint/node/automodeService.ts | Implements per-conversation routing cache, adds routing method plumb-through, and introduces post-summarization re-routing API + telemetry. |
| src/platform/configuration/common/configurationService.ts | Adds experiment-based config keys for per-conversation routing and routing-method selection. |
| src/extension/prompts/node/agent/summarizedConversationHistory.tsx | Adds optional post-summary callback hook (and an exclude flag) to trigger re-routing after summarization. |
| src/extension/prompts/node/agent/agentPrompt.tsx | Plumbs resolveNextEndpoint into summarized history rendering. |
| src/extension/intents/node/agentIntent.ts | Wires post-summarization callback to call AutomodeService.resolveEndpointForSummarization() and passes it to foreground/background compaction. |
| src/extension/intents/node/editCodeIntent2.ts | Updates constructor injection / super-call ordering to pass IAutomodeService. |
| src/extension/intents/node/askAgentIntent.ts | Updates constructor injection / super-call ordering to pass IAutomodeService. |
| src/extension/intents/node/notebookEditorIntent.ts | Updates constructor injection / super-call ordering to pass IAutomodeService. |
Comments suppressed due to low confidence (1)
src/platform/endpoint/node/automodeService.ts:503
resolveEndpointForSummarizationupdates the cachedendpoint, but does not update/clearconversationRoutedModel. If per-conversation routing is enabled, the next turn will still reuse the original first-turn model and can immediately undo the post-summarization switch. Update the cache entry so subsequent turns follow the post-summarization decision (e.g., setconversationRoutedModelto the new model, or clear it to force re-routing).
this._autoModelCache.set(conversationId, {
...entry,
endpoint: autoEndpoint,
lastSessionToken: token.session_token,
});
src/extension/prompts/node/agent/summarizedConversationHistory.tsx
Outdated
Show resolved
Hide resolved
Add copilotchat.autoModeRoutingMethod ECS flight to control which routing algorithm the auto-intent-service uses per-request: '' (empty/default) = use server default 'binary' = binary classifier v1 (path B) 'hydra' = HYDRA multi-head capability matching (path C) Changes: - configurationService.ts: new AutoModeRoutingMethod exp-based setting - routerDecisionFetcher.ts: add routing_method to request body + response type - automodeService.ts: read exp var in per-turn routing, handle fallback=true Related: github/auto-intent-service aashnagarg/per-request-routing
9268cd0 to
88f7436
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
copilotchat.autoModeRoutingMethodECS flight variable to control which routing algorithm the auto-intent-service uses per-request. This enables A/B/C experiments for the HYDRA model router.A/B/C Experiment Design
''(empty, default)'binary''hydra'Fallback for B/C → automod (path A), signaled via
fallback=truein the response.Changes
AutoModeRoutingMethodexperiment-based settingroutingMethodparam togetRouterDecision()routing_methodin request body when set by ECSRouterDecisionResponsewith new server fields:routing_method,fallback,fallback_reason,hydra_scores,chosen_model,chosen_shortfallAutoModeRoutingMethodexp var in per-turn routing callfallback=trueresponses (fall back to automod default selection)Related
github/auto-intent-servicebranchaashnagarg/per-request-routinggithub/copilot-apiPRaashnagarg/routing-method-tests