Commit 5be1449
committed
Containerize vllm-benchmark + sglang benchmarks for OSDC (all devices)
Run every matrix device inside a job-level container: instead of the old
'docker run + docker exec' pattern, since OSDC/ARC pods have no Docker daemon.
- generate_vllm_benchmark_matrix.py: emit a per-entry 'device-name' so the
workflow can resolve the container image up front (regenerated the test
fixture, which also clears pre-existing config drift).
- set-parameters: resolve the upstream image on ubuntu-latest (which has a
daemon) via 'docker manifest inspect', then enrich every matrix entry with
container-image + device-appropriate container-options. sglang resolves per
image suffix (cuda / -cu128-b200 / -rocm630-mi30x) and skips non-cuda/rocm
devices instead of failing the whole matrix.
- benchmarks job: add container: { image, options }, drop the device probe
(device-name comes from the matrix) while keeping the runtime DEVICE_TYPE
detection, run the benchmark script natively, and assume the upload IAM
role via OIDC for all devices (no host instance role inside a pod).
chown is made sudo-optional for the in-container root user.
Flagged for CI validation: the per-model S3 'already benchmarked' dedup is
dropped (needs the runtime device-type before the container exists); the
rocm/hpu container options are best-effort; and --shm-size may be capped on
ARC pods.1 parent ebadce7 commit 5be1449
5 files changed
Lines changed: 1265 additions & 512 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
314 | 314 | | |
315 | 315 | | |
316 | 316 | | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
317 | 321 | | |
318 | 322 | | |
319 | 323 | | |
| |||
0 commit comments