Skip to content

Commit b2b4762

Browse files
s0lCodex
andcommitted
docs: track MTP optimization debt
Signed-off-by: s0l <thetrues0l@gmail.com> Co-authored-by: Codex <codex@local>
1 parent 5bf4932 commit b2b4762

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

docs/design/qwen36_tp3_dcp_research.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,20 @@ clean no-MTP baseline is the `52-54 tok/s` warmed sequential result. The next
226226
step is a controlled MTP A/B: matched no-MTP vs MTP runs over 10-20 warmed
227227
sequential requests at 256 and 512 generated tokens, then concurrency 4 and 8.
228228

229+
If the first controlled MTP result is slower than no-MTP, that should be treated
230+
as optimization debt rather than a reason to drop MTP. The draft path still
231+
represents one of the main remaining acceleration levers. Work items include:
232+
233+
- measure acceptance by prompt class and generated-token length;
234+
- sweep `num_speculative_tokens` instead of assuming `1` is best;
235+
- reduce or shard the replicated MTP `fc` memory cost for TP3;
236+
- check whether MTP improves aggregate throughput at concurrency even when a
237+
single request is slower;
238+
- tune CUDA graph and FlashInfer warmup coverage for MTP shapes.
239+
240+
The target is to turn MTP from "works on TP3" into an additional speedup layer
241+
on top of the already-working TP3/DCP3 long-context baseline.
242+
229243
## DFlash Findings
230244

231245
DFlash is attractive because it drafts a whole token block in one pass instead

0 commit comments

Comments
 (0)