Skip to content

Add routing_method experiment variable for A/B/C hydra experiment#4947

Open
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/hydra-routing-method
Open

Add routing_method experiment variable for A/B/C hydra experiment#4947
aashna wants to merge 1 commit intomicrosoft:mainfrom
aashna:aashnagarg/hydra-routing-method

Conversation

@aashna
Copy link
Copy Markdown
Contributor

@aashna aashna commented Apr 2, 2026

Summary

Add copilotchat.autoModeRoutingMethod ECS flight variable to control which routing algorithm the auto-intent-service uses per-request. This enables A/B/C experiments for the HYDRA model router.

A/B/C Experiment Design

Path ECS Value Behavior
A '' (empty, default) Production automod — no router call
B 'binary' Binary classifier v1 (current router)
C 'hydra' HYDRA multi-head capability matching (new)

Fallback for B/C → automod (path A), signaled via fallback=true in the response.

Changes

  • configurationService.ts: New AutoModeRoutingMethod experiment-based setting
  • routerDecisionFetcher.ts:
    • Add routingMethod param to getRouterDecision()
    • Send routing_method in request body when set by ECS
    • Update RouterDecisionResponse with new server fields: routing_method, fallback, fallback_reason, hydra_scores, chosen_model, chosen_shortfall
    • Add routing method + fallback info to trace log and telemetry
  • automodeService.ts:
    • Read AutoModeRoutingMethod exp var in per-turn routing call
    • Handle fallback=true responses (fall back to automod default selection)

Related

  • Server-side: github/auto-intent-service branch aashnagarg/per-request-routing
  • Proxy tests: github/copilot-api PR aashnagarg/routing-method-tests

Copilot AI review requested due to automatic review settings April 2, 2026 19:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds richer Auto Mode model-routing behavior (including HYDRA routing metadata) and wires post-summarization re-routing so the system can switch models at a compaction boundary.

Changes:

  • Adds per-conversation routing (route once on first turn, reuse thereafter) plus related cache/telemetry in AutomodeService.
  • Extends router decision shape to include HYDRA/fallback metadata and introduces an experiment/config key for routing method selection.
  • Wires an optional post-summarization callback through the agent prompt/summarizer flow to allow re-routing after a summary is produced.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/platform/endpoint/node/test/automodeService.spec.ts Adds unit tests for per-conversation routing vs per-turn routing behavior.
src/platform/endpoint/node/routerDecisionFetcher.ts Extends router response typing/logging and adds an (intended) routing method parameter.
src/platform/endpoint/node/automodeService.ts Implements per-conversation routing cache, adds routing method plumb-through, and introduces post-summarization re-routing API + telemetry.
src/platform/configuration/common/configurationService.ts Adds experiment-based config keys for per-conversation routing and routing-method selection.
src/extension/prompts/node/agent/summarizedConversationHistory.tsx Adds optional post-summary callback hook (and an exclude flag) to trigger re-routing after summarization.
src/extension/prompts/node/agent/agentPrompt.tsx Plumbs resolveNextEndpoint into summarized history rendering.
src/extension/intents/node/agentIntent.ts Wires post-summarization callback to call AutomodeService.resolveEndpointForSummarization() and passes it to foreground/background compaction.
src/extension/intents/node/editCodeIntent2.ts Updates constructor injection / super-call ordering to pass IAutomodeService.
src/extension/intents/node/askAgentIntent.ts Updates constructor injection / super-call ordering to pass IAutomodeService.
src/extension/intents/node/notebookEditorIntent.ts Updates constructor injection / super-call ordering to pass IAutomodeService.
Comments suppressed due to low confidence (1)

src/platform/endpoint/node/automodeService.ts:503

  • resolveEndpointForSummarization updates the cached endpoint, but does not update/clear conversationRoutedModel. If per-conversation routing is enabled, the next turn will still reuse the original first-turn model and can immediately undo the post-summarization switch. Update the cache entry so subsequent turns follow the post-summarization decision (e.g., set conversationRoutedModel to the new model, or clear it to force re-routing).
			this._autoModelCache.set(conversationId, {
				...entry,
				endpoint: autoEndpoint,
				lastSessionToken: token.session_token,
			});

Add copilotchat.autoModeRoutingMethod ECS flight to control which routing
algorithm the auto-intent-service uses per-request:
  '' (empty/default) = use server default
  'binary' = binary classifier v1 (path B)
  'hydra' = HYDRA multi-head capability matching (path C)

Changes:
- configurationService.ts: new AutoModeRoutingMethod exp-based setting
- routerDecisionFetcher.ts: add routing_method to request body + response type
- automodeService.ts: read exp var in per-turn routing, handle fallback=true

Related: github/auto-intent-service aashnagarg/per-request-routing
@aashna aashna force-pushed the aashnagarg/hydra-routing-method branch from 9268cd0 to 88f7436 Compare April 2, 2026 20:53
@aashna aashna changed the title Aashnagarg/hydra routing method Add routing_method experiment variable for A/B/C hydra experiment Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants