Skip to content

[full_dtensor] Allow sharding config to wrap declared plain inputs#3860

Open
Matrix-Z97 wants to merge 1 commit into
pytorch:mainfrom
Matrix-Z97:full-dtensor-input-wrap
Open

[full_dtensor] Allow sharding config to wrap declared plain inputs#3860
Matrix-Z97 wants to merge 1 commit into
pytorch:mainfrom
Matrix-Z97:full-dtensor-input-wrap

Conversation

@Matrix-Z97

@Matrix-Z97 Matrix-Z97 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR updates Module._redistribute_inputs() so plain Tensor inputs with explicit in_src_shardings are wrapped via DTensor.from_local() before the full_dtensor plain-Tensor guard is applied.

With this change:

  • plain Tensor inputs declared in in_src_shardings are converted according to their sharding config;
  • undeclared plain Tensor inputs still raise in full_dtensor mode;
  • existing DTensor placement validation remains unchanged.

Why

The existing ShardingConfig contract documents that in_src_shardings can be used to annotate plain tensors as DTensors at module boundaries.

However, the previous full_dtensor logic rejected any plain Tensor input before checking whether that input had a declared source sharding. As a result, this documented path was unreachable in full_dtensor mode:

plain Tensor + in_src_shardings -> DTensor.from_local(...)

I am not entirely sure whether the previous behavior was intentional or a bug. The current ShardingConfig and _redistribute_inputs() documentation both suggest that plain Tensor inputs with explicit in_src_shardings should be wrapped at the module boundary, but full_dtensor mode previously raised before that path could run. Could reviewers confirm whether this relaxation matches the intended full_dtensor contract?

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jul 3, 2026
@pytorch-bot

pytorch-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.

@fegin

fegin commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

For full dtensor mode, the inputs shouldn't have plain tensors. The raise is intentional, not a miss. in_src_shardings for plain tensor use case is primarily preserved for the legacy mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/8gpu CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants