File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -226,6 +226,20 @@ clean no-MTP baseline is the `52-54 tok/s` warmed sequential result. The next
226226step is a controlled MTP A/B: matched no-MTP vs MTP runs over 10-20 warmed
227227sequential requests at 256 and 512 generated tokens, then concurrency 4 and 8.
228228
229+ If the first controlled MTP result is slower than no-MTP, that should be treated
230+ as optimization debt rather than a reason to drop MTP. The draft path still
231+ represents one of the main remaining acceleration levers. Work items include:
232+
233+ - measure acceptance by prompt class and generated-token length;
234+ - sweep ` num_speculative_tokens ` instead of assuming ` 1 ` is best;
235+ - reduce or shard the replicated MTP ` fc ` memory cost for TP3;
236+ - check whether MTP improves aggregate throughput at concurrency even when a
237+ single request is slower;
238+ - tune CUDA graph and FlashInfer warmup coverage for MTP shapes.
239+
240+ The target is to turn MTP from "works on TP3" into an additional speedup layer
241+ on top of the already-working TP3/DCP3 long-context baseline.
242+
229243## DFlash Findings
230244
231245DFlash is attractive because it drafts a whole token block in one pass instead
You can’t perform that action at this time.
0 commit comments