Skip to content

fix(ckpt): backport attention backend checkpoint inference to r0.5.0#4379

Open
cuichenx wants to merge 1 commit into
r0.5.0from
chcui/cp-4370-r0.5.0
Open

fix(ckpt): backport attention backend checkpoint inference to r0.5.0#4379
cuichenx wants to merge 1 commit into
r0.5.0from
chcui/cp-4370-r0.5.0

Conversation

@cuichenx

Copy link
Copy Markdown
Contributor

Summary

Conflict resolution

The auto cherry-pick conflicted because tests/unit_tests/scripts/test_text_generation.py does not exist on r0.5.0. I added a narrow release-compatible test file containing the #4370 attention-backend regression coverage, without main-only distributed-env tests.

Testing

  • git diff --check --cached
  • python3 -m py_compile scripts/inference/text_generation.py tests/unit_tests/scripts/test_text_generation.py
  • uv run python -m pytest tests/unit_tests/scripts/test_text_generation.py -q (blocked locally: nvidia-resiliency-ext==0.6.0 has no compatible wheel for this host)
  • uv run pre-commit run --all-files (blocked locally: same nvidia-resiliency-ext wheel/platform issue)

Cherry-pick of #4370.

Signed-off-by: Chen Cui <chcui@nvidia.com>
(cherry picked from commit f817629)
@cuichenx cuichenx added cherry-pick Run CICD bug Something isn't working area:ckpt Checkpoint conversion, loading, export, and save paths labels Jun 16, 2026
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Light Code ReviewClean cherry-pick of #4370 to r0.5.0. The refactoring extracts _build_megatron_checkpoint_overrides and adds the missing attention_backend key to the Megatron checkpoint override dictionary. No logic errors or typos found.__Test coverage__The single test (test_megatron_checkpoint_overrides_preserve_attention_backend) covers the happy path where all optional fields are present. This is adequate for the targeted regression fix, but note that the negative paths are untested in this backport:- attention_backend=None (key should be absent from overrides)- Provider without cache_mla_latents attribute (key should be absent)- inference_moe_token_dispatcher_type=None (key should be absent)- dtype=torch.float16 (should produce bf16=False, fp16=True)These are low-risk since the conditionals are straightforward, and this is a release backport -- not the place to expand coverage.__Suggested test cases__No perf tests impacted.

@yaoyu-33 yaoyu-33 added needs-review PR is ready for code review and waiting on a reviewer r0.5.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. labels Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ckpt Checkpoint conversion, loading, export, and save paths bug Something isn't working cherry-pick needs-review PR is ready for code review and waiting on a reviewer r0.5.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. Run CICD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants