Skip to content

feat: skip NVFP4 compression on meta-device tensors - #810

Open
kylesayrs wants to merge 1 commit into
mainfrom
nvfp4-meta-tensor-pathway
Open

feat: skip NVFP4 compression on meta-device tensors#810
kylesayrs wants to merge 1 commit into
mainfrom
nvfp4-meta-tensor-pathway

Conversation

@kylesayrs

Copy link
Copy Markdown
Collaborator

Summary

  • Adds _skip_meta_device decorator to NVFP4PackedCompressor.compress() that short-circuits quantize+pack when the weight tensor is on the meta device
  • Constructs correctly-shaped meta placeholders (weight_packed as uint8, weight_scale cast to scale_dtype) instead of running actual computation
  • Enables resumable model-free PTQ runs where already-processed shards are loaded as meta tensors for weight map / total_size accounting

Companion PR: llm-compressor (TBD) adds the resume/skip pathway in process_file()

Test plan

  • Verify NVFP4 compression still works normally (non-meta tensors)
  • Verify meta-device tensors produce correct placeholder shapes
  • Verify MXFP4 inherits the decorator correctly (group_size=32 path)

🤖 Generated with Claude Code

Add _skip_meta_device decorator to NVFP4PackedCompressor.compress()
that short-circuits quantize+pack when the weight tensor is on the meta
device. Instead, it constructs correctly-shaped meta placeholders
(weight_packed as uint8, weight_scale cast to scale_dtype). This enables
resumable model-free PTQ runs where already-processed shards are loaded
as meta tensors for accounting without re-computing compression.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e27e5578-3cd0-4850-9966-bf50ba1049ae

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mergify

mergify Bot commented Aug 17, 2026

Copy link
Copy Markdown

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require one maintainer review 👀 reviews

🔴 Require one maintainer review

Waiting for any of

  • approved-reviews-by=HDCharles
  • approved-reviews-by=brian-dellabetta
  • approved-reviews-by=dsikka
  • approved-reviews-by=kylesayrs
This rule is failing.

All PRs must have at least one approving review from a maintainer before merging.

  • any of:
    • approved-reviews-by=HDCharles
    • approved-reviews-by=brian-dellabetta
    • approved-reviews-by=dsikka
    • approved-reviews-by=kylesayrs
  • #changes-requested-reviews-by = 0

@mergify

mergify Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @kylesayrs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant