perf(ep): optimize InterNodeV1LL small-token latency on EP16 - #613

Draft

isytwu wants to merge 9 commits into

mainfrom

perf/ep16-kernel-opt-final

Commits on Aug 31, 2026

perf(ep): trim per-call host overhead on the dispatch/combine path
isytwu
committed
8ee2580
perf(ep): use the 16B load-first gather in the v1_ll combine
isytwu
committed
e147684
fix(profiler): use the intranode_ll slot enum in intranode_ll.hpp
isytwu
committed
464dfde
perf(profiler): decode trace buffers with numpy
isytwu
committed
37ea75a
tools(profiler): aggregate EP kernel traces across ranks
isytwu
committed
ce03c19
tune(ep): add mi308x InterNodeV1LL ep16 rules for hidden 6144
isytwu
committed
4e0577e
fix(ep): get the real stream pointer for kernel launch, not its packed id
isytwu
committed
8a1be90
tune(ep): re-tune MI300X EP16 v1_ll at hidden 6144, tokens 4/8/16/32
isytwu
committed
a9116d0
feat(bench): socket bootstrap for shmem p2p benchmarks (no MPI)

jhchouuu
and
claude
committed
2fb0f87