feat: skip NVFP4 compression on meta-device tensors - #810
Conversation
Add _skip_meta_device decorator to NVFP4PackedCompressor.compress() that short-circuits quantize+pack when the weight tensor is on the meta device. Instead, it constructs correctly-shaped meta placeholders (weight_packed as uint8, weight_scale cast to scale_dtype). This enables resumable model-free PTQ runs where already-processed shards are loaded as meta tensors for accounting without re-computing compression. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews
🔴 Require one maintainer reviewWaiting for any of
This rule is failing.All PRs must have at least one approving review from a maintainer before merging.
|
|
This pull request has merge conflicts that must be resolved before it can be |
Summary
_skip_meta_devicedecorator toNVFP4PackedCompressor.compress()that short-circuits quantize+pack when the weight tensor is on the meta deviceweight_packedas uint8,weight_scalecast toscale_dtype) instead of running actual computationCompanion PR: llm-compressor (TBD) adds the resume/skip pathway in
process_file()Test plan
🤖 Generated with Claude Code