Skip to content

Commit bd8d84d

Browse files
committed
docs(flowcontrol): expand fail-closed staleness rationale in utilization README
Staleness tends to correlate with overload, so failing open would hide exactly the wrong endpoints. Also note a staleness policy can be added later without changing the default. Signed-off-by: Luke Van Drie <lukevandrie@google.com>
1 parent 236b67f commit bd8d84d

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

  • pkg/epp/framework/plugins/flowcontrol/saturationdetector/utilization

pkg/epp/framework/plugins/flowcontrol/saturationdetector/utilization/README.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,11 @@ health of model-server metrics collection (scrape path, port, TLS, auth). A flee
3030
`flow_control_pool_saturation` at 1.0 and halts dispatch entirely. The `flow_control_stale_endpoints` gauge and a
3131
rate-limited detector log distinguish this from genuine overload: stale endpoints read exactly 1.0, while genuine
3232
oversubscription typically reads above 1.0. This fail-closed posture is deliberate — admitting blind on missing
33-
data risks overloading model servers with no backpressure signal at all.
33+
data risks overloading model servers with no backpressure signal at all. Staleness also tends to correlate with
34+
overload: a server too busy to serve its metrics endpoint is often the one that is saturated, so failing open
35+
would hide exactly the wrong endpoints. The posture is not configurable; if field evidence shows a need, a
36+
staleness policy (for example, holding the last known score for a bounded window) can be added later without
37+
changing this default.
3438

3539
### Role in Scheduling (The Traffic Shaper)
3640
The detector implements the `Filter` interface to protect individual endpoints. It removes endpoints from candidate lists if their telemetry is stale, or if they exceed specific safety limits:

0 commit comments

Comments
 (0)