test(flowcontrol): eviction dynamics benchmark [stacked on #2093]#2107
Draft
LukeAVanDrie wants to merge 6 commits into
Draft
test(flowcontrol): eviction dynamics benchmark [stacked on #2093]#2107LukeAVanDrie wants to merge 6 commits into
LukeAVanDrie wants to merge 6 commits into
Conversation
…he tracker RequestEvictor gains the confirmation signal for reclamation pacing (an at-most-once listener fired when an evicted request's stream terminates), a victim-priority peek, and a strict per-decision victim priority bound on EvictN so multi-revocation decisions cannot churn the demand band itself. Signed-off-by: Luke Van Drie <lukevandrie@google.com>
… eviction Implements the controller from docs/flow-control-eviction.md (PR llm-d#2061): on HoL blocking, scan blocked bands for queued demand strictly above the victim head, size revocations as a saturation deficit against a mean-footprint credit with pending-reclaim debits, and pace by confirmation (stream termination) plus a grace covering the sensor's visibility lag, with a timeout so a hung stream cannot wedge the gate. Bands with unattainable ceilings are excluded from demand. Adds the flowControl.enableEviction API field and its translation. New metrics: revocations issued and terminal outcomes, reclaim target, pending reclaim, confirmation latency. Signed-off-by: Luke Van Drie <lukevandrie@google.com>
The confirmation grace is derived at wiring time from the selected saturation detector (dispatch-tick scale for the concurrency detector; refresh interval + staleness + engine reclaim budget for scraped sensors); remaining controller parameters are internal. The Director tracks dispatches and stream termination; the ext_proc server receives the evict-channel lookup. Signed-off-by: Luke Van Drie <lukevandrie@google.com> Co-authored-by: Rishabh Saini <rishabhsaini01@gmail.com> Signed-off-by: Luke Van Drie <lukevandrie@google.com>
Signed-off-by: Luke Van Drie <lukevandrie@google.com>
Operators tuning holdback and priorities need to see which band's demand drives destruction; the terminal-outcome counter stays unlabeled because it measures actuator health, not band economics. Adds an emission test covering the eviction metrics. Signed-off-by: Luke Van Drie <lukevandrie@google.com>
Closed-loop scenario benchmark for the reclamation controller's pacing (BenchmarkEvictionDynamics): a synthetic pool with configurable sensor lag drives the real FlowController, registry, RequestEvictor, and ReclamationController end to end. Measures time-to-first-relief, HP wait percentiles, goodput, and over-eviction against an analytic minimum, across sensor profiles, grace values, per-decision caps, burst shapes, and eviction-off baselines. Opt-in via -bench, skipped under -short; EVICTION_BENCH_TABLE emits a markdown summary and EVICTION_BENCH_FULL runs the full matrix. Reproduces the Validation results in docs/flow-control-eviction.md (PR llm-d#2061). Signed-off-by: Luke Van Drie <lukevandrie@google.com>
github-actions
Bot
added
the
size/XXL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind test
What this PR does / why we need it:
The closed-loop simulation harness behind the Validation section of the in-flight eviction design (#2061): a synthetic pool with configurable sensor lag drives the real FlowController, registry, RequestEvictor, and ReclamationController end to end. It measures time-to-first-relief, HP wait percentiles, goodput, and over-eviction against an analytic minimum, across sensor profiles, grace values, per-decision caps, burst shapes, and eviction-disabled baselines. This is the artifact that makes the design doc's quantitative claims reproducible.
Opt-in only: runs via
-bench EvictionDynamics -benchtime=1x, skipped under-short.EVICTION_BENCH_TABLE=1prints a markdown summary;EVICTION_BENCH_FULL=1runs the full matrix.Which issue(s) this PR fixes:
Part of #1119
Release note:
Test plan: