Skip to content

[ckpt] feat: support HF INT4 compressed-tensors dequantization - #60

Merged
kaimo455 merged 3 commits into
masterfrom
codex/int4-pack-dequant
May 22, 2026
Merged

[ckpt] feat: support HF INT4 compressed-tensors dequantization#60
kaimo455 merged 3 commits into
masterfrom
codex/int4-pack-dequant

Conversation

@kaimo455

@kaimo455 kaimo455 commented May 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add on-the-fly compressed-tensors INT4 dequantization for HuggingFace checkpoint loading
  • select only packed INT4 companion tensors needed by the current layer/expert slice
  • wire the new flags through checkpoint conversion args, Kimi K2.5 conversion script, and dist-checkpoint config
  • pass GITLEAKS_LICENSE into the existing gitleaks action so organization secret scanning can use the licensed action

Adapted from internal iCode review 120841416 patchset 11. The GitHub master branch already had the nested expert-id flattening helper, and docker/Dockerfile.internal no longer exists there, so those parts are naturally absent from this public diff. The CI workflow change is GitHub-only and requires the repository or organization secret GITLEAKS_LICENSE to be configured.

Verification

  • git diff --check origin/master..HEAD
  • python -m compileall -q tools/convert_checkpoint/arguments.py tools/convert_checkpoint/huggingface/compressed_tensors_dequant.py tools/convert_checkpoint/huggingface/huggingface_checkpoint.py tools/convert_checkpoint/module_convertor/model.py tools/dist_checkpoint/checkpoint/hf_checkpoint_converter.py tools/dist_checkpoint/config/parallel_config.py

Change-Id: I4774385c861289a6c74597cfdc130bd4d314f969
(cherry picked from commit 3cc0c3c)
@kaimo455 kaimo455 self-assigned this May 22, 2026
@kaimo455 kaimo455 changed the title Add HF INT4 compressed-tensors dequantization [ckpt] feat: support HF INT4 compressed-tensors dequantization May 22, 2026
@kaimo455
kaimo455 requested a review from Zachary-wW May 22, 2026 05:08
@github-actions github-actions Bot added the ci label May 22, 2026
@kaimo455
kaimo455 merged commit cb62761 into master May 22, 2026
7 of 8 checks passed
@kaimo455
kaimo455 deleted the codex/int4-pack-dequant branch May 22, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant