Skip to content

Commit 5be1449

Browse files
committed
Containerize vllm-benchmark + sglang benchmarks for OSDC (all devices)
Run every matrix device inside a job-level container: instead of the old 'docker run + docker exec' pattern, since OSDC/ARC pods have no Docker daemon. - generate_vllm_benchmark_matrix.py: emit a per-entry 'device-name' so the workflow can resolve the container image up front (regenerated the test fixture, which also clears pre-existing config drift). - set-parameters: resolve the upstream image on ubuntu-latest (which has a daemon) via 'docker manifest inspect', then enrich every matrix entry with container-image + device-appropriate container-options. sglang resolves per image suffix (cuda / -cu128-b200 / -rocm630-mi30x) and skips non-cuda/rocm devices instead of failing the whole matrix. - benchmarks job: add container: { image, options }, drop the device probe (device-name comes from the matrix) while keeping the runtime DEVICE_TYPE detection, run the benchmark script natively, and assume the upload IAM role via OIDC for all devices (no host instance role inside a pod). chown is made sudo-optional for the in-container root user. Flagged for CI validation: the per-model S3 'already benchmarked' dedup is dropped (needs the runtime device-type before the container exists); the rocm/hpu container options are best-effort; and --shm-size may be capped on ARC pods.
1 parent ebadce7 commit 5be1449

5 files changed

Lines changed: 1265 additions & 512 deletions

File tree

.github/scripts/generate_vllm_benchmark_matrix.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,10 @@ def generate_benchmark_matrix(
314314
# I opt to return a comma-separated list of models here
315315
# so that we could run multiple models on the same runner
316316
"models": model,
317+
# Emit the device so the workflow can resolve the
318+
# container image up front instead of probing the
319+
# hardware at runtime (OSDC pods run in a container).
320+
"device-name": RUNNER_TO_PLATFORM_MAPPING[runner],
317321
}
318322
)
319323

0 commit comments

Comments
 (0)