[xpu][benchmarks] Enable float8 roofline benchmarks for Intel XPU

[kgajdamo]

Summary

Enable a subset of benchmarks/float8/ scripts to run on Intel GPU (XPU) via the torch.accelerator API. CUDA behavior unchanged.

• Device-agnostic selection (torch.accelerator instead of hardcoded "cuda"); torch_accelerator alias for synchronize() / get_device_name().
• XPU profiler activity (ProfilerActivity.XPU) + Level Zero sync-event filtering;
• Added Intel Arc B580 spec to roofline_utils.py; get_roofline_gpu_name() now resolves XPU device names.
• XPU guardrails: restrict recipes to tensorwise/rowwise; NotImplementedError for MX recipes and fp8 conv (not yet supported); skip the SM-100 conv check on non-CUDA.
• Fix: add missing tensorwise scale branch in get_gemm_times() (inference roofline).

Test plan

Same commands for both backends; CUDA output unchanged:

python benchmarks/float8/bench_matmul.py --shape_gen_name llama
python benchmarks/float8/bench_matmul.py --recipe rowwise --shape_gen_name llama
python benchmarks/float8/float8_roofline.py results_tensorwise.csv --float8_recipe_name tensorwise --n_limit 1
python benchmarks/float8/float8_inference_roofline.py --recipe_name tensorwise --n_limit 1
python benchmarks/float8/float8_inference_roofline.py --recipe_name rowwise --do_benchmarks False --shape_gen_name llama

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4524

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 9c8cf0a with merge base d1e0e2d ():

NEW FAILURE - The following job has failed:

• Run 1xL4 Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre torch --index-url https://download.p... / linux-job (gh)
  RuntimeError: Command docker exec -t 2ab4042a8add4e828c8edfc2617573641aae043b282907f20f6313c5e046fd4c /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[xiaowangintel]

@claude pls follow this skills #3823 (comment) to review this pr.

[claude]

Claude finished @xiaowangintel's task in 7m 9s —— View job

---

PR Review: Enable float8 roofline benchmarks for Intel XPU

• Read the diff and changed files
• Review for correctness, XPU enablement, best practices
• Answer reviewer question re: _DEVICE in float8_roofline.py:159

Note: I couldn't open the linked skills comment (pytorch/ao#3823) or the GitHub API — WebFetch/gh api aren't permitted in this run. The review below is based on the diff. If those skills contain specific conventions you want applied, paste them and I'll re-review against them.

---

Answering @xiaowangintel: "Is _DEVICE required?" (float8_roofline.py:159)

Yes — at line 159 it replaces the previously hardcoded torch.device("cuda"), so it's needed to make get_gemm_times() device-agnostic. The module-level _DEVICE is also reused at lines 104, 114, 253, 261, so removing it isn't free.

That said, _DEVICE doesn't have to be a module-level global. The concern with computing it at import time is below (it can crash on CPU-only imports). If you'd prefer, get_gemm_times() could take device as a parameter (it's always called from run(), which already has it), avoiding the global entirely. Either is fine — but the import-time evaluation is worth addressing.

---

Main feedback

1. torch.accelerator.current_accelerator().type at module import can crash (medium)
In float8_roofline.py:75 and float8_inference_roofline.py:63, _DEVICE is evaluated at import time. torch.accelerator.current_accelerator() returns None when no accelerator is present, so .type raises AttributeError on a CPU-only machine — just importing the module fails, before any benchmark runs. bench_matmul.py:37 has the same call but inside run(), which is safer. Consider guarding:

acc = torch.accelerator.current_accelerator()
_DEVICE = acc.type if acc is not None else "cpu"

These are GPU benchmarks so it may be acceptable, but an import-time crash is an unfriendly failure mode (e.g. for doc tooling / test collection).

2. Inconsistent device-name handling across the three scripts (minor / maintainability)
float8_roofline.py and float8_inference_roofline.py use the torch_accelerator alias for get_device_name(0), but bench_matmul.py:60-63 instead does an explicit if device == "xpu" branch. Same logic, two patterns. Unifying on the torch_accelerator alias (or extracting a small helper like get_roofline_gpu_name-style resolver) would reduce duplication. Note roofline_utils.py:112-118 also re-implements this same xpu/cuda name dispatch — three copies total.

3. Arc B580 spec: fp8_peak_tops == bf16_peak_tops (please confirm)
In roofline_utils.py:100-101 both are 116.5e12. On NVIDIA/AMD entries fp8 is 2× bf16. If Battlemage XMX genuinely delivers the same throughput for fp8 as bf16, fine — but it's surprising enough to deserve a one-line comment so a future reader doesn't think it's a copy-paste error. Also worth confirming the 116.5e12 figure: Intel's spec page lists peak TOPS/TFLOPS that may already account for sparsity or a different dtype, so double-check the bf16 dense number.

4. do_matmul / scale branches assume torch._scaled_mm works on XPU
float8_inference_roofline.py:153-155 adds the tensorwise scale branch (good catch — fixes the missing case). The guardrails correctly restrict XPU to tensorwise/rowwise. Just confirm torch._scaled_mm with these scale shapes is actually supported on XPU in your torch build, since the roofline do_benchmarks=True path will call it.

5. aten::convolution_overrideable added to conv kernel filter (utils.py:478)
Good — that's the dispatch key XPU/out-of-tree backends route convs through. No issue.

6. File mode change on float8_inference_roofline.py (100644 → 100755)
The diff flips this file to executable. Likely unintentional (the others stayed 644). Suggest reverting:

git update-index --chmod=-x benchmarks/float8/float8_inference_roofline.py

---

Things that look correct ✅

• zeCommandListHostSynchronize filtering in profiler_output_to_filtered_time_by_kernel_name (utils.py:90) — correct Level-Zero analogue of the cudaDeviceSynchronize skip.
• ProfilerActivity.XPU selection and torch_accelerator.synchronize() swaps.
• The device == "cuda" guard around the SM-100 conv check (float8_inference_roofline.py:763) — correct, avoids calling is_sm_at_least_100() semantics on XPU.
• NotImplementedError guardrails for MX recipes and fp8 conv on XPU are clear and have actionable messages.
• Per-call device = args[0].device.type in get_gpu_kernel_gemm_time_s/get_gpu_kernel_conv_time_s — robust.

Suggested follow-ups

• Fix import-time crash + dedupe device-name logic →

Overall this is a clean, well-scoped enablement PR that keeps CUDA behavior unchanged. The only item I'd treat as a real bug is the import-time None crash (#1) and the likely-accidental file mode change (#6); the rest are maintainability/confirmation notes.
· branch kgajdamo/xpu-benchmarks-roofline

[pytorch-bot]

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.