[Performance]: Sequence Parallelism (SP) Validation & Optimization Tracking

[ica-chao]

Proposal to improve performance

We are temporarily pausing deep performance debugging on Sequence Parallelism due to a lack of immediate, active asks from the community for this specific feature.

How we plan to prioritize:
If you need Sequence Parallelism, please help us validate its current state:

1. Vote: Please 👍 this issue if Sequence Parallelism is a blocking requirement for your workloads.
2. Share Benchmarks: If you test SP on different models or hardware configurations (e.g., v5e, v6e), please paste your vllm bench serve results in the comments below! We need to understand if the regression noted below is specific to Qwen architectures/v7x8 topology, or if the original PR benchmarks were anomalous.

Report of performance regression

Sequence Parallelism (SP) is currently functionally implemented and passes correctness tests on the tpu-inference backend (merged via PR #520).

The original implementation reported a 6-12% throughput improvement when enabling SP. However, recent independent internal verification showed a significant performance regression compared to the non-SP baseline.

Benchmarks (Qwen/Qwen2.5-32B on v7x8):

• Without SP (Baseline): 12.65 req/s
• With SP Enabled: 10.04 req/s

Misc discussion on performance

Because of this discrepancy, we are marking SP as 🧪 Experimental (Unoptimized) in our support matrix (parallelism_support_matrix.csv) while we gather community feedback in this thread.

Your current environment (if you think it is necessary)

Hardware: v7x8
Model: Qwen/Qwen2.5-32B

Before submitting a new issue...

• Make sure you already searched for relevant issues and checked the documentation page, which can answer lots of frequently asked questions.