feat: MI300X disaggregated inference with Broadcom IBGDA (#982)#998

Open

JordanNanos wants to merge 42 commits intomainfrom

jordan/mi300x-disagg-bnxt

Commits on Apr 2, 2026

Commits on Apr 4, 2026

fix: replace srun-based IP/model checks with scontrol in job.slurm

JordanNanos
and
claude
committed

Commits on Apr 5, 2026

ci: add 2-node-only disagg config keys for MI300X

JordanNanos
and
claude
committed
fix: submit sbatch as root so srun steps work on worker nodes

JordanNanos
and
claude
committed
fix: use squeue -j JOB_ID instead of squeue -u USER for job polling

JordanNanos
and
claude
committed
fix: use sudo -E to preserve environment when submitting sbatch as root

JordanNanos
and
claude
committed
fix: sanitize MODEL_NAME in Docker container name

JordanNanos
and
claude
committed
fix: add timeout to NFS refresh srun step to prevent infinite hang

JordanNanos
and
claude
committed
fix: use timeout wrapper instead of invalid srun --timeout flag

JordanNanos
and
claude
committed
fix: increase prefill TP from 4 to 8 in 2-node configs for R1-0528

JordanNanos
and
claude
committed

Commits on Apr 6, 2026

fix: fall back to /vfs/models_blog when HF cache blobs are missing

JordanNanos
and
claude
committed
fix: auto-configure UFW to allow cross-node external IP traffic

JordanNanos
and
claude
committed
fix: bind container barrier server to 0.0.0.0 for cross-subnet access

JordanNanos
and
claude
committed
fix: handle zero TPOT in process_result.py to avoid ZeroDivisionError

JordanNanos
and
claude
committed

Commits on Apr 8, 2026

fix: address review comments on process_result.py and job.slurm

JordanNanos
and
claude
committed
Merge remote-tracking branch 'origin/main' into jordan/mi300x-disagg-bnxt
JordanNanos
committed
ci: re-trigger sweep after UFW fix on node 049

JordanNanos
and
claude
committed
ci: re-run sweep for full pareto coverage

JordanNanos
and
claude
committed
Merge remote-tracking branch 'origin/main' into jordan/mi300x-disagg-bnxt
JordanNanos
committed

Commits on Apr 9, 2026

feat: add MI300X single-node benchmarks for GLM-5, Kimi K2.5, MiniMax, Qwen

JordanNanos
and
claude
committed
fix: use semianalysiswork sgl-bnxt-cdna3 image for all MI300X SGLang configs

JordanNanos
and
claude
committed
feat: add GLM-5 FP8 MI300X single-node benchmark script

JordanNanos
and
claude
committed
fix: use sglang v0.5.10 image for GLM-5 MI300X

JordanNanos
and
claude
committed
feat: add Qwen 3.5 and GLM-5 disaggregated inference on MI300X

JordanNanos
and
claude
committed

Commits on Apr 10, 2026

feat: build and publish sgl-bnxt-cdna3:v0.5.10-bnxt for GLM-5 disagg

JordanNanos
and
claude
committed
fix: add --ep-dispatch-algorithm fake to Qwen/GLM-5 disagg configs

JordanNanos
and
claude
committed
fix: use v0.5.10-bnxt for Qwen disagg — v0.5.9 produces 0 successful requests

JordanNanos
and
claude
committed
fix: reduce memory config for Qwen/GLM-5 disagg on MI300X (192GB HBM3)

JordanNanos
and
claude
committed
fix: disable CUDA graphs and reduce memory further for Qwen/GLM-5 disagg

JordanNanos
and
claude
committed

Commits on Apr 11, 2026

fix: add librocm_smi64.so.7 compat symlink in v0.5.10-bnxt image

JordanNanos
and
claude
committed
fix: reduce decode mem_fraction to 0.5 for Qwen/GLM-5 disagg on MI300X

JordanNanos
and
claude
committed

Commits on Apr 13, 2026

feat: use AMD-built sgl-bnxt-cdna3:amd-main-bnxt image for Qwen/GLM-5 disagg

JordanNanos
and
claude
committed
fix: use amd-main-bnxt-nopatch image (fixes mistral_common ImportError)

JordanNanos
and
claude
committed

Commits on Apr 14, 2026

fix: remove MoRI transfer backend for Qwen/GLM-5 disagg

JordanNanos
and
claude
committed
fix: use private interconnect for disagg node communication

JordanNanos
and
claude
committed
fix: set SGLANG_HOST_IP to private IP for mooncake disagg transfers

JordanNanos
and
claude
committed
feat: use v0.5.10-mori-bnxt image — MoRI compiled against SGLang v0.5.10

JordanNanos
and
claude
committed