Skip to content

[BugFix] Harden memmap path handling - #1762

Merged
vmoens merged 2 commits into
pytorch:mainfrom
peterdsharpe:agent/harden-memmap-paths
Aug 9, 2026
Merged

[BugFix] Harden memmap path handling#1762
vmoens merged 2 commits into
pytorch:mainfrom
peterdsharpe:agent/harden-memmap-paths

Conversation

@peterdsharpe

Copy link
Copy Markdown
Contributor

Summary

  • Apply robust filesystem encoding consistently to nested TensorDict directories, dynamically created subtrees, and subpath resolution.
  • Retain legacy-layout fallback for safe single-component names.
  • Reserve new memory-map targets atomically, reject symbolic-link targets, and preserve the existing no-file behavior for zero-element tensors.
  • Clarify filesystem ownership and overwrite expectations in the storage documentation.

Rationale

Nested collections previously did not use the same filesystem-key handling as tensor leaves. This aligns save and load behavior across both cases while retaining the explicit robust_key=False compatibility path and existing overwrite workflows.

Validation

  • 1760 passed, 54 skipped across memmap-focused constructor, TensorDict, and round-trip tests
  • Focused robust-key, legacy-layout, symbolic-link, and key-codec checks
  • Repository-pinned ufmt, flake8, and pydocstyle checks
  • RST-title and diff validation

@github-actions github-actions Bot added the bug Something isn't working label Aug 8, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 8, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation Test memmap MemoryMappedTensor labels Aug 8, 2026

@vmoens vmoens left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!
Can you have a look at 0211cce before I merge?

@vmoens
vmoens marked this pull request as ready for review August 8, 2026 20:07
@peterdsharpe

Copy link
Copy Markdown
Contributor Author

LGTM thanks! Can you have a look at 0211cce before I merge?

Good catch! Yep, LGTM!

@vmoens
vmoens merged commit e70b891 into pytorch:main Aug 9, 2026
69 of 71 checks passed
@peterdsharpe
peterdsharpe deleted the agent/harden-memmap-paths branch August 9, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation memmap MemoryMappedTensor Test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants