Skip to content

Commit d59d301

Browse files
committed
refactor(disaggregation): gating skip on ANY strict header, not just [0]
Reviewer flagged the [0] shortcut on filter.go's header-pin check: Selectors[0] being "the revision axis" is an implicit derivation (revisionLabelKey = Selectors[0].LabelKey inside newController), not anything the config surface names. Leaking that convention into the gating fast path made the code fragile — a config that puts a non- revision selector first would silently break the check. Simpler and safer rule: any strict pin from the client means "I've expressed a constraint, respect it." Skip gating and let the strict filter do the narrowing. The picker then distributes over what survives — no stochastic revision guess that could conflict with the pin and 503. Bonus: this handles cross-axis edge cases better. If a request pins `slice=s2` only, and one revision has no s2 pods: - previous rule: gating fires, weight-picks a revision, strict then narrows within it — if the picked revision has no s2 pods, 503. - new rule: gating skips, strict narrows all pods to slice=s2 (the revision with no s2 pods drops out naturally), picker uniform. Zero 503. Rename: hasRevisionHeader → hasStrictHeader (matches the new logic). Signed-off-by: Mathis Felardos <mathis@mistral.ai>
1 parent 391cb75 commit d59d301

1 file changed

Lines changed: 17 additions & 16 deletions

File tree

pkg/epp/disaggregation/filter.go

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -105,8 +105,11 @@ func (f *gatingFilter) Filter(ctx context.Context, request *fwksched.InferenceRe
105105
return append(make([]fwksched.Endpoint, 0, len(pods)), pods...)
106106
}
107107

108-
// Fast path: header-pinned. Strict downstream will do the narrowing.
109-
if f.hasRevisionHeader(request) {
108+
// Fast path: any strict pin means the client has expressed a constraint.
109+
// Strict downstream will narrow accordingly and the picker will
110+
// distribute over what survives — no reason for gating to make a
111+
// stochastic revision guess that could conflict with the pin.
112+
if f.hasStrictHeader(request) {
110113
return append(make([]fwksched.Endpoint, 0, len(pods)), pods...)
111114
}
112115

@@ -173,24 +176,22 @@ func crossRoleWeight(perRole map[string]int, required []string) int {
173176
return total
174177
}
175178

176-
// hasRevisionHeader reports whether the request carries the header that
177-
// pins the revision axis. Convention: the revision axis is the first
178-
// entry in Selectors — same source as controller.revisionLabelKey — and
179-
// only counts when it's a strict-mode selector (prefer mode is a hint,
180-
// not a pin).
181-
func (f *gatingFilter) hasRevisionHeader(request *fwksched.InferenceRequest) bool {
179+
// hasStrictHeader reports whether the request carries the header for any
180+
// strict-mode selector. Prefer-mode selectors are hints and don't count —
181+
// only a hard pin should suppress gating's weighted pick.
182+
func (f *gatingFilter) hasStrictHeader(request *fwksched.InferenceRequest) bool {
182183
if request == nil {
183184
return false
184185
}
185-
selectors := f.controller.config.Selectors
186-
if len(selectors) == 0 {
187-
return false
188-
}
189-
revSelector := selectors[0]
190-
if revSelector.Mode != ModeStrict {
191-
return false
186+
for _, selector := range f.controller.config.Selectors {
187+
if selector.Mode != ModeStrict {
188+
continue
189+
}
190+
if request.Headers[selector.HeaderName] != "" {
191+
return true
192+
}
192193
}
193-
return request.Headers[revSelector.HeaderName] != ""
194+
return false
194195
}
195196

196197
// uniqueRevisions returns the set of distinct revision-label values in

0 commit comments

Comments
 (0)