Skip to content

Commit ab6b971

Browse files
committed
README: level-3 GPU results — NodeState signal chain out of the simulated column
The reporter DaemonSet now feeds NodeState.status from real hardware (NVML + vLLM prefix-cache scrape, cross-source tokensPerJoule), so the disclosure section no longer lists that reporting as seeded — the seeding survives only in the kind rehearsals. Roadmap bullet updated with the level-3 outcome: mechanism validated on 3x A10 across 8 reps, primary hypothesis still open pending a placement-following routing layer, with the reasoning linked to the experiment design amendment.
1 parent d362a17 commit ab6b971

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -120,9 +120,9 @@ The full preemption rehearsal (A/B/C topology, load, notice injection, analysis)
120120

121121
This section stays honest about boundaries, because the value is in what is actually exercised.
122122

123-
**Real and measured:** the control plane end-to-end (reconcile, server-side apply, GC, status machines), the fleet orchestration under preemption (measured on 3x A10 above), real vLLM serving with GPU scheduling (`runtimeClassName`, digest-pinned image, model cache on hostPath) — validated both on this fleet and on GKE with GPU node pools in a separate GitOps deployment.
123+
**Real and measured:** the control plane end-to-end (reconcile, server-side apply, GC, status machines), the fleet orchestration under preemption (measured on 3x A10 above), real vLLM serving with GPU scheduling (`runtimeClassName`, digest-pinned image, model cache on hostPath) — validated both on this fleet and on GKE with GPU node pools in a separate GitOps deployment. It also covers the per-node signal chain feeding `NodeState.status`: NVML energy and utilization plus vLLM prefix-cache scrape, joined into `tokensPerJoule` on the same reporting round, exercised in vivo on 3x A10 across 8 reps ([evidence](hack/gpu-session/adr0007-ea-experiment/runs/20260723T175747)).
124124

125-
**Simulated:** the preemption *notice* (status patch, as disclosed above) and the `NodeState` warmth/utilization reporting, which in these runs was seeded rather than fed by a node agent. Both are input boundaries: everything downstream of them is real.
125+
**Simulated:** the preemption *notice* (status patch, as disclosed above). It is an input boundary: everything downstream of it is real. Earlier runs also seeded `NodeState` warmth and utilization by hand; the level-3 session replaced that with the reporter DaemonSet reading real hardware, so the seeding survives only in the kind rehearsals, where it is deterministic by design.
126126

127127
## Testing & CI
128128

@@ -133,7 +133,7 @@ This section stays honest about boundaries, because the value is in what is actu
133133

134134
- Disaggregation-aware orchestration on the Gateway API Inference Extension contract: single InferencePool v1 + `llm-d.ai/role` labels, per-role warmth semantics (decode `CacheWarm` from `vllm:prefix_cache_*`), role-differentiated placement and recovery. Design accepted in [ADR-0006](docs/adr/0006-disaggregation-aware-orchestration.md); implementation scheduled next.
135135
- Real preemption-notice detection (cloud metadata endpoint) behind the same `NodeState` interface.
136-
- Efficiency-aware closed-loop placement: the per-node reporter publishes window-based tokens/joule and KV-cache hit-rate from inferscope into `NodeState.status`; placement ranks them as strict lexicographic tie-breakers inside warmth classes (`EfficiencyAware` strategy). Design accepted in [ADR-0007](docs/adr/0007-efficiency-aware-closed-loop-placement.md). Phase A done: `EfficiencyAware` comparator as a pure function with property-tested warmth dominance and fail-open absence semantics, status fields and CRDs in place. Phase B done: per-node reporter (same image, DaemonSet entrypoint, disjoint-ownership status merge-patches), strategy threaded through both natural decision points (initial planning and post-preemption replacement), opt-in Helm wiring with least-privilege RBAC. Falsification level 2 passed 5/5 on kind (`hack/rehearsal/adr0007-run.sh`): EA discriminates on kvCacheHitRate at equal warmth on both decision paths, and degrades to warmth-first when signals are absent. The first in-cluster run also surfaced (and fixed) a real schema/design mismatch — required status fields are incompatible with multi-writer merge-patch ownership. Next: real signal source (NVML via inferscope + vLLM prefix-cache scrape) and GPU validation (level 3), post-2026-08-11.
136+
- Efficiency-aware closed-loop placement: the per-node reporter publishes window-based tokens/joule and KV-cache hit-rate from inferscope into `NodeState.status`; placement ranks them as strict lexicographic tie-breakers inside warmth classes (`EfficiencyAware` strategy). Design accepted in [ADR-0007](docs/adr/0007-efficiency-aware-closed-loop-placement.md). Phase A done: `EfficiencyAware` comparator as a pure function with property-tested warmth dominance and fail-open absence semantics, status fields and CRDs in place. Phase B done: per-node reporter (same image, DaemonSet entrypoint, disjoint-ownership status merge-patches), strategy threaded through both natural decision points (initial planning and post-preemption replacement), opt-in Helm wiring with least-privilege RBAC. Falsification level 2 passed 5/5 on kind (`hack/rehearsal/adr0007-run.sh`): EA discriminates on kvCacheHitRate at equal warmth on both decision paths, and degrades to warmth-first when signals are absent. The first in-cluster run also surfaced (and fixed) a real schema/design mismatch — required status fields are incompatible with multi-writer merge-patch ownership. Phase B extended with the real signal source: `VllmScrape` (vLLM prefix-cache counters, ADR-011 schema) and an NVML sampler behind a build-feature plus runtime double gate, joined cross-source into `tokensPerJoule` on the same reporting round. Level-3 mechanism validated on 3x A10 (Lambda, k3s, one vLLM per node, [evidence](hack/gpu-session/adr0007-ea-experiment/runs/20260723T175747)): NVML and scrape signals populate `NodeState.status` in vivo on all three nodes, `EfficiencyAware` and `WarmthFirst` diverge deterministically at both decision points, 8/8 reps clean with the operator log free of anything but two named benign classes. The primary hypothesis — that EA yields higher fleet hit-rate and tokens/joule — remains open: the replay drives one fixed endpoint by design and nothing routes traffic to the service just placed, so the fleet aggregate cannot carry a strategy signal (measured deltas and the reasoning are recorded as a post-run amendment in the [experiment design](hack/gpu-session/adr0007-ea-experiment/DESIGN.md)). Testing it needs a placement-following routing layer — scoped as separate work, not a longer re-run.
137137
- Scale-down orphan handling and Helm chart alignment (RBAC/CRDs) for the fleet path.
138138
- Capstone write-up linking probe, operator, and the measured preemption data.
139139

0 commit comments

Comments
 (0)