findings(p2p): GLM crossover bracketed - measured tie at 13.6K tokens

Bracket sweep at 8K/13.6K/21.6K actual tokens: recompute wins at 8K (+69%),
dead tie at 13,648, pull wins from ~22K (-35%) - the crossover is measured,
not extrapolated, and minCachedTokenDelta 16384 sits just above it. Cold-mesh
control: the first pull pays ~6.6s session establishment, which explains the
earlier single-point pull-proof reading 3.91s at ~24K; all curve points are
steady-state.

Signed-off-by: nilig <nili.ifergan@gmail.com>

Author: nilig

test/p2p-findings/SIG-REPORT.md

@@ -53,15 +53,22 @@ on-rig calibration points).
 | | | 32,768 | 983.0 ms | 376.3 ms | **-62%** |
 | | | 49,152 | 1,695 ms | 550.5 ms | **-68%** |
 | Qwen3-30B-A3B | (MoE, A3B) | 8,192 | 1,210 ms | 74 ms | **-94%** |
-| GLM-5.2-FP8 | ~93 KB | 34,214 | 4,511 ms | 2,512 ms | **-44%** |
+| GLM-5.2-FP8 | ~93 KB | 8,070 | 996 ms | 1,688 ms | +69% |
+| | | 13,648 | 1,738 ms | 1,760 ms | **+1% — crossover** |
+| | | 21,617 | 2,760 ms | 1,796 ms | **-35%** |
+| | | 34,214 | 4,511 ms | 2,512 ms | **-44%** |
 | | | 48,109 | 6,378 ms | 1,983 ms | **-69%** |
 | | | 65,111 | 8,784 ms | 1,980 ms | **-78%** |
 | | | 98,220 | 13,747 ms | 2,292 ms | **-83%** |
 
-(GLM sweep: every pull verified byte-exact from engine counters — 3.2-9.1 GB
+(GLM sweep: every pull verified byte-exact from engine counters — 0.75-9.1 GB
 loaded per point, matching tokens x 93 KB within 1%. The pull time is nearly
-flat, ~2.2 s (session floor + ~4.5 GB/s effective transfer), while recompute
-scales at ~144 us/token — so the gap grows without bound with prefix length.)
+flat, ~1.7-2.3 s (session floor + ~4.5 GB/s effective transfer), while
+recompute scales at ~130-144 us/token — so the gap grows without bound with
+prefix length. One transient to know: the *first* pull on a cold transfer
+mesh pays ~5-6 s of session establishment (measured 6.6 s for a 0.19 GB
+warm-up pull); all sweep points and all fleet results are steady-state,
+warm-mesh numbers.)
 
 The pattern is the same on every model: **below a crossover length recompute
 wins (the pull's fixed setup cost dominates); above it the pull wins and the
@@ -76,13 +83,13 @@ this table says a pull wins:
 | Llama-3.1-8B | ~2K tokens | 2,048 |
 | gpt-oss-120b | < 2K (pull wins at every measured length) | 2,048 |
 | Qwen3-30B-A3B | ~760 tokens (pull overhead ~30 ms) | 1,024 |
-| GLM-5.2-FP8 | ~15K tokens (pull floor ~2.2 s / recompute 144 us per token) | 16,384 — sits on the measured crossover |
+| GLM-5.2-FP8 | **~13.6K tokens, measured tie** (1.74 s vs 1.76 s at 13,648) | 16,384 — just above the measured crossover |
 
 The GLM threshold was first set as an engineering bracket (2,048 measurably
-fired net-loss pulls under load; a single ~24K calibration point won) and the
-length sweep afterwards confirmed it: the measured crossover, pull-floor
-divided by per-token recompute cost, lands at ~15K tokens — within 10% of the
-16,384 in production use.
+fired net-loss pulls under load) and the length sweep then bracketed and hit
+the crossover directly: recompute wins at 8K (+69%), dead tie at 13.6K, pull
+wins from ~22K on (-35%) — so `minCachedTokenDelta: 16384` fires pulls only
+in the measured-win region, with margin.
 
 Every fleet-level delta in section 2 is built on these per-miss economics:
 the router fires a pull only when a peer out-caches the scheduled pod by at

test/p2p-findings/configs/glm-5.2-p2p/README.md

@@ -95,18 +95,25 @@ byte-exact from B's `kv_offload_load_bytes_total` (loaded GB = tokens x
 
 | prompt tokens | recompute | pull | delta | KV moved |
 |---|---|---|---|---|
+| 8,070 | 1.00 s | 1.69 s | +69% | 0.75 GB |
+| 13,648 | 1.74 s | 1.76 s | +1% (tie) | 1.26 GB |
+| 21,617 | 2.76 s | 1.80 s | -35% | 2.0 GB |
 | 34,214 | 4.51 s | 2.51 s | -44% | 3.2 GB |
 | 48,109 | 6.38 s | 1.98 s | -69% | 4.5 GB |
 | 65,111 | 8.78 s | 1.98 s | -78% | 6.0 GB |
 | 98,220 | 13.75 s | 2.29 s | -83% | 9.1 GB |
 
-Pull time is nearly flat (~2.2 s session floor, ~4.5 GB/s effective);
-recompute scales at ~144 us/token. Crossover = 2.2 s / 144 us = **~15K
-tokens**, validating `minCachedTokenDelta: 16384`. Control
-(`crossover-sweep-v1-nopull-control.tsv`): the identical sweep *without* the
-injected p2p block never pulls (pull time = recompute time, 0 bytes loaded)
-— the engine does not fetch from peers autonomously; the EPP/sidecar
-directive is what triggers a pull.
+Pull time is nearly flat (~1.7-2.3 s session floor, ~4.5 GB/s effective);
+recompute scales at ~130-144 us/token. **Measured crossover: dead tie at
+13,648 tokens**; `minCachedTokenDelta: 16384` sits just above it, so fired
+pulls are always in the win region. Two controls: (1)
+`crossover-sweep-v1-nopull-control.tsv` — the identical sweep *without* the
+injected p2p block never pulls (pull time = recompute time, 0 bytes loaded):
+the engine does not fetch from peers autonomously; the EPP/sidecar directive
+is what triggers a pull. (2) The *first* pull on a cold transfer mesh pays
+~5-6 s of session establishment (6.6 s measured for a 0.19 GB warm-up pull;
+`crossover-sweep-bracket.tsv` MESHWARM line) — single-point pull-proofs on a
+fresh fleet measure this transient, not the steady-state pull.
 
 ## Findings
 

test/p2p-findings/configs/glm-5.2-p2p/crossover-sweep-bracket.log

@@ -0,0 +1,6 @@
+[06:48:28] source engine A podIP=10.0.15.53 (rank0, p2p port 7777)
+[06:48:53] MESHWARM first-pull=6.559251s loaded=0.19GB (cold-mesh datum)
+[06:49:10] L=8150 tokens=8070 cold=0.996114s warm=0.307271s pull=1.687986s Bload=0.75GB(exp 0.75) PULL_OK
+[06:49:28] L=15200 tokens=13648 cold=1.737579s warm=0.360100s pull=1.760498s Bload=1.26GB(exp 1.27) PULL_OK
+[06:49:49] L=24000 tokens=21617 cold=2.759991s warm=0.248784s pull=1.796221s Bload=2.00GB(exp 2.01) PULL_OK
+[06:49:49] SWEEP3 COMPLETE

test/p2p-findings/configs/glm-5.2-p2p/crossover-sweep-bracket.tsv

@@ -0,0 +1,4 @@
+L_target	prompt_tokens	cold_s	warm_s	pull_s	B_load_GB	verdict
+8150	8070	0.996114	0.307271	1.687986	0.75	PULL_OK
+15200	13648	1.737579	0.360100	1.760498	1.26	PULL_OK
+24000	21617	2.759991	0.248784	1.796221	2.00	PULL_OK