You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
README: level-3 GPU results — NodeState signal chain out of the simulated column
The reporter DaemonSet now feeds NodeState.status from real hardware
(NVML + vLLM prefix-cache scrape, cross-source tokensPerJoule), so the
disclosure section no longer lists that reporting as seeded — the
seeding survives only in the kind rehearsals. Roadmap bullet updated
with the level-3 outcome: mechanism validated on 3x A10 across 8 reps,
primary hypothesis still open pending a placement-following routing
layer, with the reasoning linked to the experiment design amendment.
Copy file name to clipboardExpand all lines: README.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -120,9 +120,9 @@ The full preemption rehearsal (A/B/C topology, load, notice injection, analysis)
120
120
121
121
This section stays honest about boundaries, because the value is in what is actually exercised.
122
122
123
-
**Real and measured:** the control plane end-to-end (reconcile, server-side apply, GC, status machines), the fleet orchestration under preemption (measured on 3x A10 above), real vLLM serving with GPU scheduling (`runtimeClassName`, digest-pinned image, model cache on hostPath) — validated both on this fleet and on GKE with GPU node pools in a separate GitOps deployment.
123
+
**Real and measured:** the control plane end-to-end (reconcile, server-side apply, GC, status machines), the fleet orchestration under preemption (measured on 3x A10 above), real vLLM serving with GPU scheduling (`runtimeClassName`, digest-pinned image, model cache on hostPath) — validated both on this fleet and on GKE with GPU node pools in a separate GitOps deployment. It also covers the per-node signal chain feeding `NodeState.status`: NVML energy and utilization plus vLLM prefix-cache scrape, joined into `tokensPerJoule` on the same reporting round, exercised in vivo on 3x A10 across 8 reps ([evidence](hack/gpu-session/adr0007-ea-experiment/runs/20260723T175747)).
124
124
125
-
**Simulated:** the preemption *notice* (status patch, as disclosed above) and the `NodeState` warmth/utilization reporting, which in these runs was seeded rather than fed by a node agent. Both are input boundaries: everything downstream of them is real.
125
+
**Simulated:** the preemption *notice* (status patch, as disclosed above). It is an input boundary: everything downstream of it is real. Earlier runs also seeded `NodeState` warmth and utilization by hand; the level-3 session replaced that with the reporter DaemonSet reading real hardware, so the seeding survives only in the kind rehearsals, where it is deterministic by design.
126
126
127
127
## Testing & CI
128
128
@@ -133,7 +133,7 @@ This section stays honest about boundaries, because the value is in what is actu
133
133
134
134
- Disaggregation-aware orchestration on the Gateway API Inference Extension contract: single InferencePool v1 + `llm-d.ai/role` labels, per-role warmth semantics (decode `CacheWarm` from `vllm:prefix_cache_*`), role-differentiated placement and recovery. Design accepted in [ADR-0006](docs/adr/0006-disaggregation-aware-orchestration.md); implementation scheduled next.
135
135
- Real preemption-notice detection (cloud metadata endpoint) behind the same `NodeState` interface.
136
-
- Efficiency-aware closed-loop placement: the per-node reporter publishes window-based tokens/joule and KV-cache hit-rate from inferscope into `NodeState.status`; placement ranks them as strict lexicographic tie-breakers inside warmth classes (`EfficiencyAware` strategy). Design accepted in [ADR-0007](docs/adr/0007-efficiency-aware-closed-loop-placement.md). Phase A done: `EfficiencyAware` comparator as a pure function with property-tested warmth dominance and fail-open absence semantics, status fields and CRDs in place. Phase B done: per-node reporter (same image, DaemonSet entrypoint, disjoint-ownership status merge-patches), strategy threaded through both natural decision points (initial planning and post-preemption replacement), opt-in Helm wiring with least-privilege RBAC. Falsification level 2 passed 5/5 on kind (`hack/rehearsal/adr0007-run.sh`): EA discriminates on kvCacheHitRate at equal warmth on both decision paths, and degrades to warmth-first when signals are absent. The first in-cluster run also surfaced (and fixed) a real schema/design mismatch — required status fields are incompatible with multi-writer merge-patch ownership. Next: real signal source (NVML via inferscope + vLLM prefix-cache scrape) and GPU validation (level 3), post-2026-08-11.
136
+
- Efficiency-aware closed-loop placement: the per-node reporter publishes window-based tokens/joule and KV-cache hit-rate from inferscope into `NodeState.status`; placement ranks them as strict lexicographic tie-breakers inside warmth classes (`EfficiencyAware` strategy). Design accepted in [ADR-0007](docs/adr/0007-efficiency-aware-closed-loop-placement.md). Phase A done: `EfficiencyAware` comparator as a pure function with property-tested warmth dominance and fail-open absence semantics, status fields and CRDs in place. Phase B done: per-node reporter (same image, DaemonSet entrypoint, disjoint-ownership status merge-patches), strategy threaded through both natural decision points (initial planning and post-preemption replacement), opt-in Helm wiring with least-privilege RBAC. Falsification level 2 passed 5/5 on kind (`hack/rehearsal/adr0007-run.sh`): EA discriminates on kvCacheHitRate at equal warmth on both decision paths, and degrades to warmth-first when signals are absent. The first in-cluster run also surfaced (and fixed) a real schema/design mismatch — required status fields are incompatible with multi-writer merge-patch ownership. Phase B extended with the real signal source: `VllmScrape` (vLLM prefix-cache counters, ADR-011 schema) and an NVML sampler behind a build-feature plus runtime double gate, joined cross-source into `tokensPerJoule` on the same reporting round. Level-3 mechanism validated on 3x A10 (Lambda, k3s, one vLLM per node, [evidence](hack/gpu-session/adr0007-ea-experiment/runs/20260723T175747)): NVML and scrape signals populate `NodeState.status` in vivo on all three nodes, `EfficiencyAware` and `WarmthFirst` diverge deterministically at both decision points, 8/8 reps clean with the operator log free of anything but two named benign classes. The primary hypothesis — that EA yields higher fleet hit-rate and tokens/joule — remains open: the replay drives one fixed endpoint by design and nothing routes traffic to the service just placed, so the fleet aggregate cannot carry a strategy signal (measured deltas and the reasoning are recorded as a post-run amendment in the [experiment design](hack/gpu-session/adr0007-ea-experiment/DESIGN.md)). Testing it needs a placement-following routing layer — scoped as separate work, not a longer re-run.
137
137
- Scale-down orphan handling and Helm chart alignment (RBAC/CRDs) for the fleet path.
138
138
- Capstone write-up linking probe, operator, and the measured preemption data.
0 commit comments