test(inference): smoke-test FP8 mixed-precision diffusion steps - #226
Conversation
Add test_nano_fp8_mixed_precision_inference to the Nano inference smoke suite: the sharded FP8 run from test_nano_fp8_inference plus --mixed-precision-first-steps=2 / --mixed-precision-last-steps=2 (the feature from #217). Asserts the exact per-step precision schedule from the MIXED_PRECISION_TRACE log line (2x W8A16 / 6x W8A8 / 2x W8A16 on the 10-step run) so a run where the flags silently never engaged fails, plus the usual non-degenerate-video check. Runs automatically in the existing generator-inference-smoke CI job (which invokes the whole file); timeout bumped 90 -> 105 minutes for the extra full-width FP8 run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Not introduced by this PR — flagging it here because it bounds how much the FP8 coverage, including the new mixed-precision case, can be trusted over time. All three FP8 cases The workflow comment already says to grep the log for Cheap fix if it seems worth it: have the Worth noting this resolves itself once the checkpoint is published under its own name and registered in |
… without the checkpoint Per review: the three FP8 cases skip when nvidia/Cosmos3-Experimental is unreachable for a credentials reason, which is right for forks and local runs but let the generator-inference-smoke job silently drop from three full-width FP8 runs to zero while staying green (token rotation, repo permissions change). That job now sets REQUIRE_FP8=1, which _download_fp8_checkpoint uses to promote the skip to a pytest.fail. Everything else keeps the skip behaviour. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Good call — implemented in 04d77a2: |
lfengad
left a comment
There was a problem hiding this comment.
Approving. Verified on the latest commit (04d77a28): all 9 checks green, and generator-inference-smoke genuinely covered FP8 rather than skipping — 5 passed, three Swapped 504 linears, and MIXED_PRECISION_TRACE steps=W8A16,W8A16,W8A8,W8A8,W8A8,W8A8,W8A8,W8A8,W8A16,W8A16 on the new case, matching the asserted 2/6/2 schedule exactly.
Thanks for picking up the REQUIRE_FP8 point — with it set on that job, a green run now proves FP8 coverage instead of relying on someone remembering to grep the log. That run also happened to hit a cold cache (41m56s vs 12m03s), which is a useful confirmation that the 90 → 105 timeout bump was needed.
What
Adds
test_nano_fp8_mixed_precision_inferencetotests/nano_inference_smoke_test.py: the same sharded-layout text2video run astest_nano_fp8_inference, plus the FP8 mixed-precision diffusion-step flags from #217 (--mixed-precision-first-steps=2 --mixed-precision-last-steps=2, defaultw8a16-cache=none— the only mode valid under FSDP sharding, which is exactly the layout used).Pass criterion
The schedule itself, not just a valid video: the test parses the
MIXED_PRECISION_TRACElog line and compares it exactly against the expected2x W8A16 / 6x W8A8 / 2x W8A16sequence for the 10-step run (num_stepsis read back from the shared_FP8_GENERATION_ARGSso the expectation cannot drift). A run where the flags silently never engaged (all-W8A8 trace or no trace) fails even though its video would look fine._assert_video_has_contentstill catches the numerically-broken-but-running case, and the existing FP8 swap-count guard confirms the FP8 path engaged at all.CI wiring
No new job or trigger: the existing
generator-inference-smokejob runs the whole file, so the case is picked up automatically. Like the other two FP8 cases it SKIPs (not fails) whenHF_TOKENcannot readnvidia/Cosmos3-Experimental. Job timeout bumped 90 → 105 min for the extra full-width run.Verification
pytest --collect-onlyinside the inference container collects 5 tests including the new case; the GPU run itself is exercised by this PR's owngenerator-inference-smokejob.🤖 Generated with Claude Code