Skip fp16-config tests on accelerators without fp16 support - #8398
Conversation
test_multiple_models/test_zero_optimizer and the MoE tests hardcode
"fp16": {"enabled": True} in their DeepSpeed configs. The engine's
sanity check then raises "Type fp16 is not supported on your device"
on any accelerator whose is_fp16_supported() is false - on CPU that
depends on the host's AVX512-FP16 capability, and GitHub's ubuntu-24.04
runners are heterogeneous enough that the same test passes on one
runner and fails on the next.
Skip these tests via a capability query instead of depending on runner
hardware, mirroring the existing bf16 skip precedent in
test_zero_user_backward.
Validated as part of the multi-rank CPU experiment in deepspeedai#8381: 146
failures of this class became deterministic skips, zero regressions.
Signed-off-by: Guokai Ma <guokai.ma@intel.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31e794e4ed
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| import deepspeed | ||
| import deepspeed.comm as dist | ||
| import torch | ||
| from deepspeed import get_accelerator |
There was a problem hiding this comment.
This is a non-merge commit, but its message has no Signed-off-by trailer. Add the author name and email from the Git configuration using git commit --signoff so the commit satisfies the repository's mandatory DCO/CI requirement.
AGENTS.md reference: AGENTS.md:L8-L8
Useful? React with 👍 / 👎.
2 participants
Problem
TestMultipleModels::test_zero_optimizer,TestSimpleMoE,TestMoE,TestPRMoE, andTestMOETensorParallelhardcode"fp16": {"enabled": True}in their DeepSpeed configs. The engine's sanity check then raises:on any accelerator whose
is_fp16_supported()is false. On CPU that maps to the AVX512-FP16 capability of the host, and GitHub'subuntu-24.04runners are hardware-heterogeneous: the same test passes on one runner and fails on the next (observed directly in #8381 — 146 failures appeared on one runner generation and none on another, with identical code).Change
Skip these tests via a capability query:
@pytest.mark.skipif(not get_accelerator().is_fp16_supported(), reason="fp16 is not supported on this accelerator")tests/unit/v1/zero/test_zero_user_backward.py;Validation
Validated as part of the multi-rank CPU CI experiment in #8381: the 146 hardware-lottery failures became deterministic skips, zero regressions on previously-passing tests.