[gfx950][k3][dspark] Improve long context decoed cost on low acceptance

[raikonenfnu]

Summary

On 8×MI350X, Kimi-K3 DSpark improves 4k/1k TP8/EP8 throughput by 39–106%, but at 131,072-token input its average accepted length collapses to exactly 1.0, turning the extra 7 draft positions plus 8-wide target verification into overhead and making decode 28–41% slower than ordinary TokenSpeed.

The first-principles performance target is: make one DSpark verification round a first-class vectorized M=8 execution path whose cost approaches one ordinary decode round, independent of how many candidates are eventually accepted. Acceptance determines how many output tokens amortize that round, but inefficient infrastructure currently inflates the numerator by 2.39×; that underlying cost should be fixed before relying on an acceptance-based fallback.

Reproduction contract

• TokenSpeed: 8911ce4cd7bd9f4fcea33bc069a5d0f27cebfa95
• Hardware: 8×AMD Instinct MI350X (gfx950)
• Model: Kimi-K3 DSpark checkpoint
• Topology: TP8/EP8
• Runtime: FP8 KV cache, graph-compiled decode, eager/chunked prefill, greedy sampling
• Workload: identical random-token prompts, exact ISL/OSL, prefix caching explicitly disabled
• “132K” below means ISL=131,072 and OSL=1,024 (132,096 total tokens/request)

Current end-to-end behavior

Shape	Non-DSpark output tok/s	DSpark output tok/s	Delta	Non-DSpark TPOT	DSpark TPOT	DSpark / base TPOT
4K/1K B1/C1	66.30	136.28	+105.6%	14.62 ms	6.83 ms	0.47×
4K/1K B8/C8	223.59	348.55	+55.9%	32.97 ms	18.74 ms	0.57×
4K/1K B16/C16	331.84	459.90	+38.6%	42.76 ms	27.64 ms	0.65×
132K/1K B1/C1	30.45	17.88	-41.3%	18.04 ms	40.69 ms	2.26×
132K/1K B2/C2	36.55	23.65	-35.3%	24.96 ms	54.03 ms	2.16×
132K/1K B4/C4	43.44	31.12	-28.4%	27.29 ms	65.57 ms	2.40×

The 132K B1 TTFT is nearly unchanged (15.14 s base versus 15.60 s DSpark), so this is a decode regression, not a long-prefill regression. Long B8 DSpark is omitted because its smaller KV pool admitted only part of the requested concurrency.

At 4K, the server reports accepted lengths of roughly 4–7 tokens, enough to repay the wider work. At 132K B1, the log reports:

Accept_num_tokens_avg: 1.0

First-64-decode-step kernel breakdown

The table sums rank-0 GPU kernel durations over the captured decode window. These are dispatch sums for attribution, not wall time: kernels/collectives may overlap or wait, and rank 0 may not be the critical rank.

Component / exact kernel	Non-DSpark calls	Base total	DSpark calls	DSpark total	DSpark/base	DSpark share
TP collective/residual: iris_stage_one_shot_allreduce_residual_attnres_gluon_kernel.kd (base) vs iris_stage_one_shot_allreduce_kernel.kd (DSpark)	5,355	279.32 ms	6,784	721.92 ms	2.58×	20.22%
MoE symmetric reduction: iris_reduce_symmetric_gluon_kernel.kd	5,888	220.72 ms	5,888	602.30 ms	2.73×	16.87%
MoE W13 + SiTU: _stage1_a16w4_situ_warp_gemv.kd	5,888	76.78 ms	5,888	410.79 ms	5.35×	11.50%
MLA decode: _mla_decode_gluon.kd	1,536	232.20 ms	1,856	217.00 ms	0.93×	6.08%
MoE W2/combine: _stage2_a16w4_warp_gemv_combine.kd	5,888	54.88 ms	5,888	191.26 ms	3.49×	5.36%
MoE routing: _sigmoid_bias_topk_route_gluon_kernel.kd	—	outside base top 10	5,796	179.12 ms	—	5.02%
Dense/projection GEMM: Cijk_...MT32x16x512...ISA950...kd	—	comparable base GEMM family 64.63 ms	11,754	132.90 ms	—	3.72%
Attention residual: _attn_res_rmsnorm_kernel.kd	structurally fused/different base path	—	11,968	116.97 ms	—	3.28%
MoE projection: _packed_projection_gemm_kernel.kd	—	outside base top 10	5,888	115.09 ms	—	3.22%
KDA: fused_recurrent_kda_mtp_fwd_kernel.kd	different base kernel	—	4,416	107.12 ms	—	3.00%
All rank-0 decode dispatch	—	1,491.90 ms	—	3,571.08 ms	2.39×	100%

The three largest regressions are especially revealing:

TP collective/residual: +442.60 ms
MoE symmetric reduction: +381.58 ms
MoE W13 + SiTU:          +334.01 ms
MoE W2/combine:          +136.37 ms

Those four matched groups explain about 62% of the extra rank-0 dispatch time. MLA is not the first target: despite more calls, its total time is slightly lower in this capture.

The kernel shape, not merely launch count, is hurting. W13 and W2 have the same number of calls in both modes, but mean call time grows from 13.04 to 69.77 µs and from 9.32 to 32.48 µs respectively. The TP path also changes from a fused allreduce_residual_attnres kernel to a plain all-reduce plus separate residual work.

Separate infrastructure efficiency from acceptance

For draft width k and accepted length a, DSpark performance decomposes into two independent quantities:

speculative_round_cost
  = draft_cost(k) + target_verify_cost(k) + routing/state overhead

infrastructure efficiency
  = speculative_round_cost / ordinary_decode_round_cost

algorithmic efficiency
  = accepted_tokens / speculative_round

DSpark cost per output token
  = speculative_round_cost / accepted_tokens

The current long-context result has poor values for both dimensions: acceptance is 1.0, while rank-0 dispatch work per profiled window is 2.39× ordinary decode. We should optimize and report the round-cost ratio even under forced rejection, so good acceptance cannot hide inefficient kernels and bad acceptance cannot hide infrastructure improvements.

At acceptance 1.0, DSpark necessarily performs some extra draft/state work, so exact parity is not guaranteed. However, K3 decode is largely weight-, memory-, and synchronization-bound: an M=8 target pass should reuse fixed weights, share KV-cache tiles across query rows, and aggregate collectives rather than paying close to eight independent token costs. The current 2.39× ratio—and especially the 2.58–5.35× growth in dominant components—leaves substantial non-fundamental overhead to remove.

Proposed fix, in priority order

P0 — make the whole M=8 verification round a first-class execution path

Treat the candidate-token dimension as a small batch carried end-to-end through routing, expert computation, collectives, residuals, and state updates—not as eight expensive variations of M=1 decode:

candidate rows
    │
    ▼
one vectorized routing/packing pass
    │
    ▼
occupancy-aware grouped MoE with shared weight loads
    │
    ▼
one collective carrying all candidate rows
    │
    ▼
fused residual + state/cache update

• Profile a single speculative round and compare it directly with a single ordinary round at identical context and active request count.
• Track useful bytes versus actual HBM bytes for fixed projections, KV-cache reads, and routed experts. Fixed weights and KV tiles should be reused across candidate rows rather than reloaded per row.
• Track dispatches per round, intermediate tensor materializations, and the slowest rank's critical path.
• Preserve the candidate dimension through operators; do not repeatedly flatten, copy, repad, or re-route it between layers.

The initial milestone is ≤1.5× ordinary round wall time under forced acceptance=1; the stretch target is ≤1.2×. These are infrastructure targets, not claims that low-acceptance speculation will outperform ordinary decoding.

P1 — make M=8 collective/residual work resemble the base fused path

• Recover the producer-direct allreduce + residual + AttentionRes fusion for the DSpark M dimension instead of using plain iris_stage_one_shot_allreduce_kernel followed by separate residual kernels.
• Measure per-rank producer-arrival and Iris wait time. If the collective duration is mostly waiting for MoE, fix the producer imbalance before tuning the reduction loop.
• Preserve vectorized M rows and avoid materializing intermediate [M, hidden] tensors between target verification stages.

This attacks the 721.92 ms TP path and 602.30 ms MoE reduction path, but must be judged on critical-path wall time across all ranks, not rank-0 summed duration.

P2 — add an M=8-aware routed-expert schedule and reuse weights

The existing warp-GEMV path scales poorly for the verified rows: W13 grows 5.35× and W2 3.49× with unchanged call count. Bucket expert work by actual token occupancy and use small-M grouped tiles when an expert receives multiple verified rows, while retaining the fast M=1 kernel for singleton experts. Load an expert's packed weights/scales once for all of its candidate rows where occupancy allows it, and fuse SiTU and weighted-combine epilogues only where doing so shortens time-to-last-expert.

Also record selected expert IDs per layer/rank. A slow rank or a few hot experts can extend both the MoE kernel tail and the following Iris wait.

P3 — vectorize routing, projections, KV access, and state updates

• Fuse or share route preparation across the draft/verify stages where semantics allow it.
• Remove copies/layout materializations around _packed_projection_gemm_kernel.
• Tile MLA over candidate queries so long-context KV tiles can be reused across rows rather than streamed independently where the attention semantics permit it.
• Batch KDA/recurrent-state and page-table updates across candidates, then commit only the accepted prefix without rebuilding the candidate state one token at a time.
• Tune graph buckets at the real active-row count and preserve vectorized candidate rows rather than padding or decomposing them into M=1 operations.

The current MLA total is already slightly lower in the captured DSpark window, so validate its HBM traffic before changing it; the immediate evidence points more strongly to MoE and collectives.

P4 — separately diagnose the acceptance collapse

Low acceptance can be legitimate for some workloads, but 1.0 at 131K is suspicious enough to investigate independently:

• Sweep 4K/32K/64K/96K/131K with the same prompts and record per-position acceptance histograms, not only one average.
• Compare draft and target logits/tokens at the first rejected position.
• Audit long-position IDs/rotary state, draft-layer KV offsets/page tables, recurrent KDA state injection, and cache updates after partial acceptance.
• Repeat with natural-text prompts; fixed random-token prompts may be out-of-distribution for the drafter.
• Distinguish checkpoint-quality degradation from an implementation defect before changing model semantics.

Acceptance investigation is deliberately separate from infrastructure optimization. We should be able to show that the DSpark round became cheaper even if this particular random-token workload continues to accept only one token.

P5 — add fallback only as a production guardrail

After the underlying path is efficient, an optional cost-aware controller can avoid inherently unprofitable speculation. This is not the main performance fix and must not conceal an acceptance or state-management bug.

If implemented, pre-capture ordinary and useful DSpark graph widths, compare measured round cost per accepted token, use hysteresis, and keep profitable requests separate from fallback requests when batching permits. Do not hard-code “disable after 128K”; context length is at most a cold-start prior because acceptance depends on the request and decode position.

Validation gates

1. Forced-low-acceptance infrastructure test: deterministically reject every draft continuation but keep M=8 verification enabled. Report DSpark-round/ordinary-round wall time, dispatch count, HBM bytes, and slowest-rank time. First target ≤1.5×; stretch ≤1.2×, down from the current 2.39× rank-0 dispatch ratio.
2. Per-component gates: reduce TP collective/residual, MoE reduction, W13, and W2 ratios from 2.58×/2.73×/5.35×/3.49× while preserving output. Report both summed dispatch and non-overlapping critical-path changes.
3. Context sweep: 4K/32K/64K/96K/131K, B1, 1K output, three runs each; publish acceptance histogram, round-cost ratio, TPOT, and output tok/s as separate columns.
4. Batch sweep: exact 131K/1K at B1/B2/B4. Report infrastructure ratio independently from the resulting acceptance-amortized TPOT.
5. Protect the win: exact 4K/1K B1/B8/B16 must retain at least 95% of current DSpark throughput (136.28/348.55/459.90 tok/s).
6. Profile correctly: unprofiled full-1K runs decide performance; separate first-64-step traces explain it. Capture every rank and report the slowest-rank critical path plus kernel dispatch sums.
7. Correctness: run Kimi-K3 AIME26 and targeted draft/target logit comparisons through rejection, partial acceptance, and full acceptance.
8. Capacity: do not report a requested batch when DSpark's draft-layer KV allocation queues some requests; report admitted concurrency explicitly.

Benchmark caveats and artifacts

• Non-DSpark exposes 1,897,600 cache tokens; DSpark exposes 1,106,560 because five draft layers also consume cache.
• Prefix caching was disabled. A fixed-seed control with caching reused up to 130,944 prompt tokens and is excluded.
• The current-main AMD benchmark required the same temporary vendor-neutral mla_kv_pack_quantize_fp8 fallback in both modes and a startup-only pageable-CPU checkpoint-loading workaround; neither is inside the timed/profiled interval.
• Local raw logs, throughput JSON, 64 rank-local Perfetto traces, and parsed kernel JSON are under /home/stwinata/k3_sgl_vs_ts_perf/artifacts/dspark_comparison_20260818/.

Related performance context: #55 (TokenSpeed TP8/EP8 vs SGLang), #57 (B1 roofline), and #58 (B8 roofline).

[raikonenfnu]

@jhinpan can you help look at this! :)

[jhinpan]

Current-main resolution update

I re-established the exact workload on current main at the campaign start (9196fa27) with three unprofiled full-1K runs per arm/shape, 30-s all-GPU idle canaries, TP8/EP8, FP8 KV, greedy, decode graphs, chunked/eager prefill, and prefix caching off.

• 4K/1K: base 71.696 tok/s, DSpark 152.949 tok/s (+113.33%), TPOT 13.523 -> 6.070 ms, acceptance 5.620879.
• exact 131072/1024: base 32.686 tok/s, DSpark 19.723 tok/s (-39.66%), TPOT 17.040 -> 36.880 ms (2.164x), acceptance 1.0.

So the issue still reproduces on an ancestor of today's main. Main advanced during the campaign to d262066f; the only directly relevant intervening change, #1156, keeps AMD on the existing scratch fallback because its replay registrations are NVIDIA-gated. Any future performance PR should still rerun this exact gate after rebasing.

Acceptance track

The collapse is both length- and distribution-dependent, not one simple page-table clamp:

• random prompts: avg acceptance 5.29/5.52 at 4K/8K, 3.02/2.95/2.89 at 49K/65K/98K, then 1.0 at exact 131K;
• natural text: 6.68 at 32K, 5.77 at 98K, 1.764 at exact 131K;
• repeated text: 7.94 at 4K/32K, 5.08 at 98K, 1.814 at exact 131K.

Fine samples from 114K through 131K were non-monotonic (1.0 to 1.7) rather than a single exact threshold. At matched ~40K final context, a 4K prompt followed by 36,864 generated tokens had 5.91 acceptance over its last 128 logged rounds, while a 40,960-token prompt + 128 output measured 6.68; that does not support a KDA commit/rollback error accumulating with decode steps at this range. RoPE capacity and runtime geometry remain above the tested position.

The null-page inconsistency is real but small: adding page_ids <= 0 to DFlash changed exact-131K acceptance from 1.0000 to 1.0365 and a single full request from 19.72 to 20.28 tok/s. It is therefore an independent correctness fix, not the root cause: #1169.

The remaining evidence points most strongly to draft long-position/full-history quality (amplified by random-token prompts). I do not see evidence for a runtime clamp/scale/state patch that would be honest to open yet; the next acceptance step should compare draft/target logits against the reference implementation/checkpoint at the first 98K-vs-131K rejection.

Infrastructure track

#1170 adds an env-gated forced-rejection control. Under exact 131K acceptance=1:

• ordinary round: 17.06 ms;
• M=8 speculative round: 36.80 ms;
• wall ratio: 2.157x (target <=1.5x).

Forced DSpark was token-identical to ordinary greedy for both a 4K/1K repeated prompt and exact-131K/128 natural text.

The all-rank first-64 trace reproduces the structural counts: 5,888 MoE reductions/W13/W2 calls, 11,968 AttnRes calls, 4,416 KDA verify calls, and 1,152 NCCL kernels (18/round, including the 14 proposal gathers). Iris time is highly rank-skewed: attention/MoE Iris sums range from 0.666/0.496 s on rank 5 to 2.624/1.382 s on rank 7. Producer arrival/wait is therefore a first-order part of the reported Iris duration.

Existing WIP exact results, posted back to their owning PRs:

• perf(comm): add two-stage producer-direct reduction #1141 standalone: engine ready, but 1-token health requests took ~4.5–5.5 s and SMG could not register the worker; not independently E2E-valid.
• perf(kimi3): tune small-batch latent projection #1135 M=8 add3 only: 36.85 ms, 19.742 tok/s versus 36.80 ms, 19.765 tok/s; flat/slightly regressed.
• perf(kimi-k3): tune small-M MoE decode #1140 M=8 router-warp line: 36.77 ms, 19.775 tok/s; ~0.05%, not material.
• perf(comm): add two-stage producer-direct reduction #1141 + perf(comm): extend fused AttnRes through M16 #1142 + (WIP) perf(kimi3): enable batched AttnRes runtime fusion #1146: three-run forced TPOT 36.25/36.25/36.25 ms, mean 19.970 tok/s (+1.04%); normal 4K DSpark 155.078 tok/s (+1.39% vs current-main baseline), and token identity passes. This is a real but small topology-specific win, moving the ratio only to ~2.125x; (WIP) perf(kimi3): enable batched AttnRes runtime fusion #1146's reported M=2/4 EP1 regressions still require shape/topology gating before landing.

No current PR fixes the whole issue. The two small runtime PRs are independently landable; the measured communication win belongs in the existing WIP stack; the dominant remaining M=8 EP8 W13/W2 weight-reuse problem is still unaddressed.

[jhinpan]

PR status against this issue, and what is still unaddressed

Re-reading the whole cross-referenced set against this issue's gates, so the ownership is explicit and nothing here is mistaken for a fix.

The kernel/comm stack is not scoped to this issue

@panditsa's stack (#1138 -> #1139 -> #1140 -> #1141 -> #1142 -> #1144 -> #1145 -> #1146) targets 4K/1K, which he confirmed in #1141: he has not been working on DSpark or 131072/1024. My measurements on that stack were therefore only a report of what it does to this issue's M=8/131K axis, not a judgement of the PRs:

• perf(kimi-k3): tune small-M MoE decode #1140's only M=8-applicable line (sigmoid-router warp count): +0.05%.
• perf(comm): add two-stage producer-direct reduction #1141 standalone: never reached a benchmarkable serving state; SMG could not register the worker.
• perf(comm): add two-stage producer-direct reduction #1141 + perf(comm): extend fused AttnRes through M16 #1142 + (WIP) perf(kimi3): enable batched AttnRes runtime fusion #1146: forced 131K TPOT 36.80 -> 36.25 ms (+1.04% throughput), moving the round ratio only 2.157x -> ~2.125x.
• perf(kimi3): tune small-batch latent projection #1135, merged as of 2026-08-24: its M=8 add3 portion measured flat/slightly regressed at 131K, which does not conflict with its 4K/1K purpose.

#1138 is now out of WIP and awaiting review, so that stack is progressing on its own axis. Nothing in it should be closed on account of this issue.

My two PRs are rebased onto current main

Both were 60 commits behind and needed re-basing before any gate rerun is meaningful:

• fix(cache): route null decode pages to dummy slot #1169 (null/negative decode page routing, independent correctness fix): cd59967b on top of 2a400090.
• feat(dspark): expose forced-rejection diagnostics #1170 (TOKENSPEED_FORCE_SINGLE_TOKEN_VERIFY, the acceptance-independent gate 1 instrument): 11ea9cb7 on top of 2a400090. One import conflict with the new ForwardThread import; both imports kept.

Local verification after rebase: test/runtime/test_decode_cache_loc.py 4 passed; isort, black, py_compile, and whitespace clean on all changed files. I could not run test_force_single_token_verify.py or test_sliding_cache_loc.py locally, because the tokenspeed_kernel wheel in my campaign venv (0.1.3.dev20260819+git8911ce4c) predates current main's kernel API. That is an environment gap on my side, not a change in the PRs.

The previous #1170 head failed eval-kimi-k3-dspark-mxfp4-tp8ep8-aime26-amd at 0.8667 against the >= 0.9 threshold, plus a b200 gpt-oss GPQA eval and three NVIDIA ARM unit tests whose logs have since expired. The same k3-dspark eval passed on the three most recent completed main runs, and a same-family eval failed on main at 1e084e57, so I cannot yet call the failure either noise or a regression. The env flag is default-off, so the rebased rerun should settle it.

Both PRs now have every workflow in action_required. I do not have the rights to approve fork-PR runs, so a maintainer approval is needed before the gates produce anything. @raikonenfnu could you approve them?

What still blocks this issue

The dominant cost is untouched. At exact 131K under forced acceptance=1, one M=8 speculative round is 36.80 ms against a 17.06 ms ordinary round, and the best measured combination moves that to ~2.125x versus the ≤1.5x first target. The unaddressed part is P2: at M=8/EP8 the warp-GEMV routed-expert path grows W13 mean call time 13.04 -> 69.77 µs and W2 9.32 -> 32.48 µs at unchanged call counts, so verified candidate rows are still paying close to per-row weight loads instead of sharing an expert's packed weights. That is where I am taking the work next, and any numbers above need re-establishing on current main after the 60-commit gap.

[jhinpan]

Acceptance collapse: infra ruled out on current main, evidence points at the draft's extrapolation range

@raikonenfnu I ran the acceptance track to a conclusion on current main (b2aa43d), 8×MI355X gfx950, TP8/EP8, B1, greedy, decode graphs, prefix caching off, --max-model-len 132096, FP8 KV unless stated, OSL=128. Every infrastructure hypothesis I could construct and test is ruled out by experiment. The remaining evidence is consistent with the DSpark draft checkpoint's own positional extrapolation limit, not with a TokenSpeed defect. One discriminator is left that I cannot settle from inside TokenSpeed, and it is the thing I would like your opinion on — it is at the end.

First, a measurement correction that affects the numbers in this issue

Acceptance varies about ±50% between prompts at a fixed length. Two same-spec requests at ISL 98,304 measured 4.870 and 3.236. Fixed-seed repeats are bit-identical, so repeating a run reports zero variance and creates false precision — that is what my earlier single-prompt points (and the ones in the issue body) were doing. Everything below uses five different prompts per length.

ISL	mean	per-prompt values	spread
32,768	3.761	4.704 / 3.432 / 2.702 / 3.735 / 4.233	2.00
98,304	2.914	3.969 / 2.268 / 2.822 / 1.984 / 3.528	1.99
106,496	1.821	1.789 / 1.896 / 1.740 / 1.841 / 1.841	0.16
122,880	1.717	1.841 / 1.740 / 1.789 / 1.649 / 1.568	0.27
131,072	1.046	1.076 / 1.124 / 1.008 / 1.000 / 1.024	0.12

The load-bearing signal is not the falling mean, it is the collapse of content dependence. Below ~100K the coefficient of variation across prompts is ~25%; from 106K up it is 3–6% and the value is pinned. A drafter that has merely become weaker keeps content dependence — easy prompts stay easy. Losing it means the draft's output has decoupled from the content it should be conditioned on.

Per-position agreement shows the same thing, and locates it before any autoregressive chaining: p1, the first draft token, falls from 0.688 at 98,304 to 0.228 at 131,072. At 98,304 the profile (0.688 0.688 0.662 0.525 0.400 0.375 0.250) is shaped like a normal drafter and is indistinguishable from 32,768; at 131,072 it is flat and low (0.228 0.174 0.174 0.186 0.132 0.186 0.192). So this is not the Markov head's chaining.

What is ruled out

Capacity-derived sizing. Doubling --max-model-len from 132,096 to 262,144 left the curve unchanged: 32,768 and 98,304 are bit-identical, 106,496 is 1.811 vs 1.821, 131,072 is 1.059 vs 1.046. Since max_draft_prefix, the staged draft page-table width, and the graph capture widths all derive from physical_context_len, that single run rules out the whole family. Note the arena is memory-sized (2240 vs 2242 parents), so pool capacity was held constant while the derived extents doubled.

Draft page-table holes and the max_draft_prefix clamp. Instrumented directly at a 131K decode step (eager, so the Python path executes; eager acceptance matches graph mode — 98,304 identical, 131,072 1.085 vs 1.076):

prefix_len=131075  clamped=False  max_draft_prefix=132153  view_max_tokens=132160
page_size=64  table_width=2065  live_pages=2049
null_pages_in_live_extent=0  first_null_index=-1
row_head=[604, 605, 606, 607]  row_tail=[6539, 6540, 6541, 6542]
block_positions=[131075, 131076, 131077, 131078, 131079, 131080, 131081]

The draft's sequence length is the full context, the clamp has 1,078 tokens of headroom and never fires, all 2,049 live pages are populated with contiguous ids, and the query positions are exact absolute positions. 98,304 is the same with prefix_len=98305, live_pages=1537, zero nulls.

This also resolves what #1169 was seeing: the null pages it guards appear only on the 1-token health probe, whose live extent legitimately has an unallocated page 0. They are absent from long-context decode. #1169 stays a real correctness fix, but it does not explain this issue and the 1.0000 -> 1.0365 shift I reported earlier was not evidence of a long-context page defect.

FP8 context injection. The concern raised in #1015 is real — the engine does log Using FP8 KV cache but no scaling factors provided for both the target and the draft load — but it is not this. With BF16 KV the collapsed points are unchanged: 106,496 1.779 vs 1.821, 131,072 1.051 vs 1.046, and content independence persists.

Draft YaRN rope numerics. DeepseekScalingRotaryEmbedding builds a float32 cos_sin_cache over the full 1,048,576 positions. Against an independent float64 DeepSeek-YaRN computation the max absolute cos/sin error is 2.0e-5 at 1,024, 1.1e-3 at 32,768, 4.3e-3 at 98,304 and 3.1e-3 at 131,072 — a smooth fp32 large-argument artifact with no discontinuity anywhere near 100K. Correction range is (15, 26), the reference mscale ratio is exactly 1.000000, and the slowest dimension's phase at 131,072 is 0.115 rad, so there is no wraparound. The config plumbing is also correct: the draft ships the newer rope_parameters key and kimi_k3_dspark_config.py maps it to deepseek_yarn with factor/original_max/beta/mscale intact, preferring the draft's own rope_theta.

Anchor or position offset. A shift scan over draft/target rows (does draft[i] equal target[i+k]) has shift 0 dominant at every length; the nonzero-shift rates are explained by repeated common tokens in random-token continuations.

What the evidence positively supports

The transition sits at ~100K of absolute context and does not move with capacity. Architecturally the draft is the only component with a positional encoding at all: the target's text_config sets mla_use_nope: true and carries no rope scaling, so its long-range behaviour comes from the KDA layers, while the draft sets mla_use_nope: false and uses YaRN with original_max_position_embeddings: 32768, factor: 32.0. That is exactly why the target is healthy at 131K while the draft is not.

And 98,304 = 3 × 32,768 is the last length that still shows content dependence. A useful range of roughly 3× the original YaRN window for a 5-layer drafter is an ordinary outcome, and it reproduces the full observed shape: gradual decline from ~100K, pinned content-independent acceptance from ~106K, worse again at 131K.

The one thing I cannot settle from here, and where I'd like your view

deepseek_v3.py multiplies the draft's attention scaling by yarn_get_mscale(32, mscale_all_dim)^2 = 1.813260. That follows DeepSeek-V3's own convention and is self-consistent with how the cos/sin mscale ratio is applied (which is 1.0 here, since the checkpoint sets mscale: 1.0 and mscale_all_dim: 1.0). What I have not verified is whether the DSpark reference implementation applies the same correction for this checkpoint. It matters because an over-sharp softmax degrades more as the key count grows: attention collapses onto a few keys, which is precisely the loss of content dependence we measure. If the reference omits that factor, the draft's attention is 1.81× too sharp in TokenSpeed and the diagnosis flips from "checkpoint limit" to "TokenSpeed modelling difference".

Three questions:

1. Do you know, or can you check, whether the reference DSpark path applies the mscale_all_dim softmax correction? No further TokenSpeed-side experiment can answer this.
2. If it does match and this is the checkpoint's extrapolation range, do you want to route to a longer-context draft checkpoint, or take the P5 cost-aware controller — noting the issue explicitly says a fallback must not conceal a bug, and we would now have the evidence that there is no bug to conceal?
3. Separately on the infra track: I measured the P2 premise directly and it does not hold. Holding the route count at 16 and varying only how many distinct experts they touch, going from 1 route/expert to 8 routes/expert buys 1.082x on W13 and 1.211x on W2 — that is the ceiling of "load an expert's weights once for all its candidate rows", in a limit real routing cannot reach. In the fully shared case the kernel still costs 4.62x of M=1 while reading the same distinct bytes, so M=8 is bound by per-route request/dequant work, not by weight traffic. Worth reconsidering whether P0–P3 stay in scope at that ceiling.

Method notes: the campaign venv needed tokenspeed-smg 1.10.0 to match main's launcher (1.9.0 rejects --disable-load-monitoring); the instrumentation above was diagnostic only and is not proposed for merge. One artifact I discarded rather than report: a 98,304-token greedy generation fell into a 3-token repeat loop, which produced a p3 match rate of exactly 0.000 over 41,897 blocks — a very convincing-looking structural defect that is only the cycle's phase, since the same position measures 0.66 in the non-degenerate runs.

Finally, #1169 and #1170 are rebased onto current main but every workflow is stuck in action_required and I do not have rights to approve fork-PR runs.