Skip to content

[test] fix: repair CP packed SFT functional setup#4374

Merged
yaoyu-33 merged 1 commit into
mainfrom
yuya/seqpacking-cp-sft-checkpoint-fix
Jun 15, 2026
Merged

[test] fix: repair CP packed SFT functional setup#4374
yaoyu-33 merged 1 commit into
mainfrom
yuya/seqpacking-cp-sft-checkpoint-fix

Conversation

@yaoyu-33

Copy link
Copy Markdown
Contributor

Summary

  • Split the CP + packed SFT functional checkpoint setup fix out from PR [test] fix: register internal pytest marker #4359 for faster review.
  • Create a tiny pretrain checkpoint first, verify it, then use it as checkpoint.pretrained_checkpoint for the SFT run.
  • Keep checkpoint.load = None so this exercises the intended finetune-from-pretrained path instead of trying to resume.

Validation

  • git diff --check HEAD~1..HEAD
  • uv tool run ruff check tests/functional_tests/test_groups/training/test_seqpacking_cp_example.py
  • uv tool run ruff format --check tests/functional_tests/test_groups/training/test_seqpacking_cp_example.py
  • uv run --no-sync python -m py_compile tests/functional_tests/test_groups/training/test_seqpacking_cp_example.py
  • uv run --no-sync pre-commit run --files tests/functional_tests/test_groups/training/test_seqpacking_cp_example.py

Exact GPU functional execution is left to CI.

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33

Copy link
Copy Markdown
Contributor Author

/ok to test dc094b0

@yaoyu-33 yaoyu-33 added area:training Training loop, callbacks, and runtime integration bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer labels Jun 15, 2026
@claude

claude Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review

LGTM. The change correctly splits the test into a two-phase pretrain → finetune flow:

  1. Creates a pretrain checkpoint via pretrain() with llama32_1b_pretrain_config
  2. Verifies the pretrain checkpoint exists
  3. Passes it as cfg.checkpoint.pretrained_checkpoint to the SFT step
  4. Sets cfg.checkpoint.load = None so SFT exercises the finetune-from-pretrained path (not resume)

Imports, config fields, and function signatures all check out.

Suggested test cases: No perf tests impacted.

@yaoyu-33 yaoyu-33 merged commit 579f5c8 into main Jun 15, 2026
20 checks passed
@yaoyu-33 yaoyu-33 deleted the yuya/seqpacking-cp-sft-checkpoint-fix branch June 15, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:training Training loop, callbacks, and runtime integration bug Something isn't working needs-review PR is ready for code review and waiting on a reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant