Skip to content

Make QuantizeTensorToFloat8Kwargs a frozen Dataclass to enable torch export#4011

Closed
asfiyab-nvidia wants to merge 1 commit intopytorch:mainfrom
asfiyab-nvidia:dev-asfiyab-enable-fp8-export
Closed

Make QuantizeTensorToFloat8Kwargs a frozen Dataclass to enable torch export#4011
asfiyab-nvidia wants to merge 1 commit intopytorch:mainfrom
asfiyab-nvidia:dev-asfiyab-enable-fp8-export

Conversation

@asfiyab-nvidia
Copy link

@asfiyab-nvidia asfiyab-nvidia commented Mar 5, 2026

Regular Dataclasses are not supported by the Dynamo Tracer as they're dispatched to UserDefinedObjectVariable which doesn't have an implementation for as_proxy. However, Frozen Dataclasses are supported by the Dynamo Tracer as the as_proxy method is defined for them (source).

Marking QuantizeTensorToFloat8Kwargs as a frozen dataclass enables torch.export on the Float8Tensor subclass.

Fixes #3928

…export

Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 5, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4011

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b491570 with merge base 5a5029d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 5, 2026
@asfiyab-nvidia
Copy link
Author

@jcaip can you please help with the review? Thanks

@jcaip jcaip added the module: inference quantize_ api inference flow label Mar 6, 2026
Copy link
Contributor

@jcaip jcaip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @asfiyab-nvidia. Can we also add an export test to test/quantization/quantize_/workflows/float8/test_float8_tensor.py so that this doesn't happen again?

@asfiyab-nvidia
Copy link
Author

@jcaip I added another comment on the issue that motivates this change. Based on the comment, would you still recommend the change the PR proposes to maintain compatibility with PyTorch < 2.7 where unwrap_tensor_subclass is neeeded?

@jcaip jcaip closed this Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: inference quantize_ api inference flow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Float8DynamicActivationFloat8WeightConfig] Enable Dynamic Quantized models to be exportable using torch.export.export

2 participants