Skip to content

Commit 3d04e73

Browse files
VincyZhangVincyZhang
authored
refactor(pd-xpu): switch PD XPU RDMA flow to kustomization (llm-d#925)
* feat(xpu): add PD XPU RDMA overlay Signed-off-by: jiafuzha <jiafu.zhang@intel.com> Signed-off-by: Zhang, Wenxin <wenxin.zhang@intel.com> * feat(pd-xpu-rdma): GPU-NIC PCIe aligned kustomization with rdma-dranet - Replace separate resource-claim-template-rdma.yaml with inline strategic merge patches in kustomization.yaml - Use deviceClassName 'rdma-dranet' (matching existing DeviceClass) - Add PCIe root alignment constraint (resource.kubernetes.io/pcieRoot) - Remove old approach of adding rdma-claim to pod spec via JSON patches - Validated: e2e 10/10 passed with NIXL UCX backend on ib,rc,ze_copy Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com> Signed-off-by: jiafuzha <jiafu.zhang@intel.com> Signed-off-by: Zhang, Wenxin <wenxin.zhang@intel.com> * docs(xpu): move XPU RDMA docs to accelerators and main PD README Remove standalone README.xpu.md and integrate its content into: - docs/accelerators/README.md: XPU RDMA prerequisites and overview - guides/pd-disaggregation/README.md: add XPU+RDMA to hardware table Signed-off-by: Zhang, Wenxin <wenxin.zhang@intel.com> --------- Signed-off-by: jiafuzha <jiafu.zhang@intel.com> Signed-off-by: Zhang, Wenxin <wenxin.zhang@intel.com> Signed-off-by: Wenxin Zhang <wenxin.zhang@intel.com>
1 parent c264fa6 commit 3d04e73

3 files changed

Lines changed: 106 additions & 0 deletions

File tree

‎docs/accelerators/README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,16 @@ Intel Data Center GPU Max 1550 and Intel BMG GPUs (Battlemage G21) are supported
6565

6666
For cluster prerequisites, ensure you have the [Intel Resource Drivers for Kubernetes](https://github.com/intel/intel-resource-drivers-for-kubernetes) installed.
6767

68+
### XPU with RDMA
69+
70+
For P/D disaggregation with RDMA-accelerated KV-cache transfer on Intel XPU, the following additional prerequisites apply:
71+
72+
- An RDMA DRA driver exposing the `rdma-dranet` device class (e.g., [rdma-dranet](https://github.com/k8snetworkplumbingwg/rdma-dra-driver)).
73+
- GPU-NIC PCIe alignment for optimal transfer performance.
74+
- UCX transport configured with `ib,rc,ze_copy`.
75+
76+
The RDMA overlay (`modelserver/xpu/vllm-rdma/`) reuses the standard XPU vLLM base and adds one RDMA DRA claim per pod plus RDMA-specific UCX transport settings. See the [P/D Disaggregation guide](../../guides/pd-disaggregation/README.md) for deployment instructions.
77+
6878
## Intel Gaudi (HPU)
6979

7080
Intel Gaudi 1, Gaudi 2, and Gaudi 3 accelerators are supported via DRA. Ensure you have the [Intel Resource Drivers for Kubernetes](https://github.com/intel/intel-resource-drivers-for-kubernetes) installed on your cluster before deploying HPU guide variants.

‎guides/pd-disaggregation/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ This guide includes configuration for the following accelerators:
4343
| Google TPU | `modelserver/tpu/vllm/` | GKE TPU, validated each release |
4444
| AMD GPU | `modelserver/amd/vllm/` | AMD GPU, community contributed |
4545
| Intel XPU | `modelserver/xpu/vllm/` | Intel Data Center GPU Max 1550+, community contributed |
46+
| Intel XPU + RDMA | `modelserver/xpu/vllm-rdma/` | Intel XPU with RDMA via UCX (`ib,rc,ze_copy`), requires RDMA DRA driver |
4647
| Intel Gaudi (HPU) | `modelserver/hpu/vllm/` | Gaudi 1/2/3 with DRA support, community contributed |
4748

4849
> [!NOTE]

‎guides/pd-disaggregation/modelserver/xpu/vllm-rdma/kustomization.yaml‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
4+
resources:
5+
- ../vllm
6+
7+
# Override the base GPU-only ResourceClaimTemplates with GPU+NIC aligned versions.
8+
# The strategic merge patches replace the devices spec to add an RDMA NIC request
9+
# and a PCIe root alignment constraint, ensuring GPU and NIC share the same
10+
# PCIe root complex for optimal RDMA transfer performance.
11+
patches:
12+
# Decode ResourceClaimTemplate: add NIC request + PCIe alignment
13+
- target:
14+
kind: ResourceClaimTemplate
15+
name: xpu-decode-claim
16+
patch: |-
17+
apiVersion: resource.k8s.io/v1
18+
kind: ResourceClaimTemplate
19+
metadata:
20+
name: xpu-decode-claim
21+
spec:
22+
spec:
23+
devices:
24+
requests:
25+
- name: gpu
26+
exactly:
27+
deviceClassName: gpu.intel.com
28+
count: 1
29+
- name: nic
30+
exactly:
31+
deviceClassName: rdma-dranet
32+
count: 1
33+
selectors:
34+
- cel:
35+
expression: device.attributes["dra.net"].rdma == true
36+
constraints:
37+
- matchAttribute: "resource.kubernetes.io/pcieRoot"
38+
# Prefill ResourceClaimTemplate: add NIC request + PCIe alignment
39+
- target:
40+
kind: ResourceClaimTemplate
41+
name: xpu-prefill-claim
42+
patch: |-
43+
apiVersion: resource.k8s.io/v1
44+
kind: ResourceClaimTemplate
45+
metadata:
46+
name: xpu-prefill-claim
47+
spec:
48+
spec:
49+
devices:
50+
requests:
51+
- name: gpu
52+
exactly:
53+
deviceClassName: gpu.intel.com
54+
count: 1
55+
- name: nic
56+
exactly:
57+
deviceClassName: rdma-dranet
58+
count: 1
59+
selectors:
60+
- cel:
61+
expression: device.attributes["dra.net"].rdma == true
62+
constraints:
63+
- matchAttribute: "resource.kubernetes.io/pcieRoot"
64+
# Decode Deployment: switch to RDMA transport and XPU kv buffer
65+
- target:
66+
kind: Deployment
67+
name: decode
68+
patch: |-
69+
- op: replace
70+
path: /spec/template/spec/containers/0/env/0/value
71+
value: ib,rc,ze_copy
72+
- op: add
73+
path: /spec/template/spec/containers/0/env/-
74+
value:
75+
name: UCX_MEMTYPE_CACHE
76+
value: "0"
77+
- op: replace
78+
path: /spec/template/spec/containers/0/args/5
79+
value: '{"kv_connector":"NixlConnector", "kv_role":"kv_both", "kv_buffer_device":"xpu"}'
80+
# Prefill Deployment: switch to RDMA transport and XPU kv buffer
81+
- target:
82+
kind: Deployment
83+
name: prefill
84+
patch: |-
85+
- op: replace
86+
path: /spec/template/spec/containers/0/env/0/value
87+
value: ib,rc,ze_copy
88+
- op: add
89+
path: /spec/template/spec/containers/0/env/-
90+
value:
91+
name: UCX_MEMTYPE_CACHE
92+
value: "0"
93+
- op: replace
94+
path: /spec/template/spec/containers/0/args/5
95+
value: '{"kv_connector":"NixlConnector", "kv_role":"kv_both", "kv_buffer_device":"xpu"}'

0 commit comments

Comments
 (0)