[EPLB] Make async EPLB default

[ilmarkov]

Set async EPLB by default

Performance evaluation

Model DeepSeekR1-NVFP4, 32xGB200, decode nodes (with 4 GPUs each), nixl eplb connector. UCX is using MNNVL. Pure decode workload: ISL=1, OSL=1500, dataset: sharegpt
Server:

vllm serve nvidia/DeepSeek-R1-0528-NVFP4-v2 --disable-uvicorn-access-log  
--tensor-parallel-size 1 --enable-expert-parallel --data-parallel-size 32 --data-parallel-rank $R 
--data-parallel-size-local 1 - --gpu-memory-utilization 0.75 --max-num-batched-tokens 1560 --max-num-seqs 1560 
--disable_cascade_attn 
--max-cudagraph-capture-size 1560 --moe-backend auto --all2all-backend deepep_low_latency --async-scheduling 
--api-server-count 4 --compilation_config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' 
--enable-eplb --eplb-config '{"window_size":100,"step_interval":1000,"num_redundant_experts":32,
"communicator":"nixl","use_async":${USE_ASYNC},"log_balancedness":true,"log_balancedness_interval":500}' 
--enable-prefix-caching --kv-cache-dtype fp8 --enable-sleep-mode --enable-force-include-usage

Results

Sync Rearrangement Stall Duration
Mean: 0.91s, Min: 0.65s, Max: 1.70s
Total stall time: ~20s across 22 rearrangements.

ITL P99

Stage	Concurrency	Async ITL P99 (ms)	Sync ITL P99 (ms)	Δ	EPLB runs (async/sync)
0	20800	76.9	74.9	~same	6 / 5
1	27200	94.8	99.0	+4%	4 / 4
2	33600	141.1	522.9	+271%	4 / 3
3	40000	148.7	203.4	+37%	2 / 3
Total					22 / 21

Async EPLB shows better p99 results without stoping the execution.

Validation

Server:

vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507-FP8 --disable-uvicorn-access-log --no-enable-prefix-caching -tp 1 -dp 4 --max-num-seqs 256 --enable-expert-parallel --port 8003 --gpu-memory-utilization 0.8 --max-model-len 4096 --async-scheduling --disable-nccl-for-dp-synchronization --all2all-backend allgather_reducescatter --enable-eplb --eplb-config.window_size 10 --eplb-config.step_interval 100 --eplb-config.num_redundant_experts 16 --eplb-config.log_balancedness_interval 128 --eplb-config.log_balancedness false --eplb-config.use_async ${USE_ASYNC}

gsm8k

default - async

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8825|±  |0.0089|
|     |       |strict-match    |     5|exact_match|↑  |0.8741|±  |0.0091|

sync EPLB:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8840|±  |0.0088|
|     |       |strict-match    |     5|exact_match|↑  |0.8772|±  |0.0090|

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[mergify]

Documentation preview: https://vllm--43219.org.readthedocs.build/en/43219/

[yewentao256]

Thanks for the work! Could you also add lm_eval metrics to show we don't hurt acc?

[gemini-code-assist]

Code Review

This pull request updates the default configuration for Expert Parallel Load Balancing (EPLB) to use asynchronous mode by setting use_async to true in vllm/config/parallel.py and the documentation. Integration tests and Buildkite configurations have been updated to explicitly use synchronous mode where necessary, with labels renamed to Sync EPLB to reflect this change. I have no feedback to provide.

[ilmarkov]

@yewentao256 The description is updated.