fix(config): auto-inject saturation detector as scheduling filter

[RishabhSaini]

Summary

Auto-inject the saturation detector as a scheduling filter into explicit scheduling profiles.

Problem

When a user defines an explicit scheduling profile, the saturation detector's Filter method is never called. Saturated endpoints remain scheduling candidates, causing hotspotting when high-weight scorers (e.g., prefix-cache-scorer at weight 3) consistently pick the overloaded pod. Flow control's HoL gate holds load near the saturation threshold, preventing the natural rebalancing that occurs without flow control.

Reported by @asm582 with prefix-cache-scorer (weight 3) + queue-scorer (weight 2) + kv-cache-utilization-scorer (weight 2) on a static two-replica system. The second pod processed zero requests with flow control enabled.

Thanks to @LukeAVanDrie for diagnosing this issue.

Fix

In ensureSaturationDetector, after the detector is resolved, check if it implements scheduling.Filter. If yes, inject it into all scheduling profiles that don't already reference it. Both utilization-detector and concurrency-detector implement Filter.

For empty profiles, the existing auto-population in ensureSchedulingLayer already includes all filters. This change closes the gap for explicit profiles.

Behavior change

Configurations with flow control enabled and an explicit scheduling profile will now have the saturation detector's filter active. Saturated pods are removed from scheduling candidates when at least one non-saturated alternative exists. When all pods exceed the threshold, the filter returns all of them (existing fallback behavior).

Test plan

• Existing config loader tests pass
• Verify with Abhishek's config: two replicas, prefix-cache-scorer weight 3, flow control enabled → both pods receive traffic

[LukeAVanDrie]

Approved with nits; thanks @RishabhSaini!

/lgtm

[LukeAVanDrie]

Non-blocking design question: this injects the detector into every profile with no opt-out. For multi-profile setups where a profile intentionally excludes saturation filtering, there's no way to decline now. Consistent with the other ensure* defaults, but unlike ensureDataLayer's InjectDefaults escape hatch.

This seems fine for the single-profile case this fixes; just flagging whether per-profile opt-out should be a follow-up.

[RishabhSaini]

filed: #1826

[ahg-g]

Are we sure this will not impact p/d?

[asm582]

@RishabhSaini @LukeAVanDrie we see the same issue when running with the optimized-baseline plugins with flow-controller enabled. Will this PR fix the issue for a different set of plugins too?

[LukeAVanDrie]

@RishabhSaini @LukeAVanDrie we see the same issue when running with the optimized-baseline plugins with flow-controller enabled. Will this PR fix the issue for a different set of plugins too?

@asm582 Can you elaborate a bit more on this? Note: this is for auto-wiring the saturation detector into a secondary role of acting as a scheduling filter (to prevent hotspotting based on per-endpoint saturation). As a workaround, you can explicitly reference the detector as a filter in the scheduling profile. The utilization-detector or concurrency-detector types implement both the SaturationDetector (primary) and Filter (secondary) extension points and are configurable as such.

This PR is to make this config more intuitive as it is non-obvious that the detector must be explicitly added as a filter too for the behavior to take effect.

Rishabh is OOO, but I think this PR is mostly ready. I'll respond to @AHG's feedback here and then provided there is no merge conflicts on rebase, we can try to get this in the next release.

[LukeAVanDrie]

Are we sure this will not impact p/d?

Yes. The filter is appended after each profile's existing filters, so it only narrows within the already role-filtered candidate set, and it can't empty a result (when all remaining candidates are saturated it returns them all, failing open). Net
effect: a saturated prefill/decode pod is skipped only when a non-saturated same-role alternative exists. If a deployment ever wants it on one role only, #1826 (per-profile opt-out) is the escape hatch.

The red pd-shared-storage-disagg check isn't this PR: the decode pod's sim (v0.9.2) panics on the new /inference/v1/generate e2e test. Main already fixed this (sim bumped to v0.10.2 in #2030); this branch's July 14 merge of main predates that. Updating the branch should turn it green.