Skip to content

Commit 6e3b62d

Browse files
committed
findings(p2p): point cross-TP open item at vllm#48414
Signed-off-by: nilig <nili.ifergan@gmail.com>
1 parent 5dd8f14 commit 6e3b62d

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

test/p2p-findings/p2p-pd-defects.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,12 @@ Open items: extending the parallel-agnostic mode to the V2 model runner and
7777
to multi-group attention layouts (the two exclusions that bit here), and -
7878
wherever a deployment falls in an excluded configuration - failing the pull
7979
cleanly (see Defect 2) and having the router avoid emitting a cross-TP
80-
source header, since the pull cannot succeed there.
80+
source header, since the pull cannot succeed there. On the first item,
81+
[vllm#48414](https://github.com/vllm-project/vllm/pull/48414) (stacked on
82+
#48408) is in flight: it stores offloaded KV in a canonical
83+
parallelism-free layout - one copy per block, no parallelism inputs in the
84+
transfer path - and fails uncertifiable configs at startup instead of
85+
silently demoting.
8186

8287
Repro: `llama-fs-crosstp.yaml` (the two TP pods) + `llama-fs-test.sh` (warm,
8388
cross-read, on-disk hash-dir dump).

0 commit comments

Comments
 (0)