[Core] Add default excludes for working_dir uploads #59566
+109
−6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
December 18, 2025 19:29
Signed-off-by: Ricardo Decal <[email protected]>
Signed-off-by: Ricardo Decal <[email protected]>
crypdick
force-pushed
the
bugfix/default-excludes-working-dir
branch
from
57c19af to
631fa2e
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a helpful feature to automatically exclude common directories like .git and venv from working_dir uploads, preventing common errors with large repositories. The implementation is clean, and it's great to see that it's accompanied by thorough documentation updates and both unit and integration tests. My only suggestion is a minor improvement to the type hinting for better code clarity.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Ricardo Decal <[email protected]>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
ray-gardener
bot
added
docs
Signed-off-by: Ricardo Decal <[email protected]>
Contributor
|
Nice nice. Just some questions: Can you clarify the following scenarios in these scenarios:
I'm wondering
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Automatically exclude common directories (.git, .venv, venv, pycache) when uploading working_dir in runtime environment packages.
At a minimum we need to exclude
.git/because unlike the others, nobody includes .git/ in.gitignore. This causes Ray to throw aray.exceptions.RuntimeEnvSetupErrorif your.gitdir is larger than 512 MiB.I also updated the documentation in handling-dependencies.rst and improved the error message if the env exceeds the GCS_STORAGE_MAX_SIZE limit.
Related issues
N/A
Additional information
This PR pytorch/tutorials#3709 was failing to run because the PyTorch tutorials .git/ folder is huge.