[core] Replace SHA-1 with SHA-256 for internal hash operations #60242

pushpavanthar · 2026-01-16T22:02:28Z

Why are these changes needed?

This PR replaces SHA-1 with SHA-256 across Ray's internal hash operations to address security concerns in #53435 with a conservative approach for runtime environment paths. SHA-1 has known collision vulnerabilities and is deprecated - switching to SHA-256 improves security and FIPS compliance.

Related issue number

Fixes #53435

SHA-256 Migration (15 files)

Function collision detection
SSH control path hashing
Configuration cache keys
CloudWatch config hashing (with method renames and UTF-8 encoding fix)
File lock paths
Deployment code versioning
Feature hashing
Test files updated

SHA-1 Retained (5 files)

Runtime environment path generation kept SHA-1 to avoid:

Path length issues on Windows and network filesystems
Breaking existing cached environments
Unnecessary environment rebuilds

For content-addressable hashing (non-cryptographic), SHA-1 provides sufficient collision resistance.

Testing

All linter checks pass
No new errors introduced
Zero breaking changes to runtime environments

gemini-code-assist

Code Review

This pull request correctly replaces SHA-1 with SHA-256 for internal hashing operations, which is a great security improvement. The changes are mostly straightforward. I've added a few comments to suggest explicitly specifying the encoding as 'utf-8' when calling .encode() on strings before hashing. This ensures consistent behavior across different platforms and avoids potential issues with default encodings.

gemini-code-assist · 2026-01-16T22:13:42Z

python/ray/_private/runtime_env/packaging.py

-    sha1 = hashlib.sha1()
-    sha1.update(str(filepath.relative_to(relative_path)).encode())
+    sha256 = hashlib.sha256()
+    sha256.update(str(filepath.relative_to(relative_path)).encode())


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

sha256.update(str(filepath.relative_to(relative_path)).encode())

sha256.update(str(filepath.relative_to(relative_path)).encode("utf-8"))

gemini-code-assist · 2026-01-16T22:13:42Z

python/ray/autoscaler/_private/command_runner.py

+        ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
+        ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

ssh_control_hash = hashlib.sha256(cluster_name.encode("utf-8")).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode("utf-8")).hexdigest()

gemini-code-assist · 2026-01-16T22:13:42Z

python/ray/tests/gcp/test_gcp_tpu_command_runner.py

+    ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
+    ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

ssh_control_hash = hashlib.sha256(cluster_name.encode("utf-8")).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode("utf-8")).hexdigest()

gemini-code-assist · 2026-01-16T22:13:42Z

python/ray/tests/gcp/test_gcp_tpu_command_runner.py

+    ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
+    ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

ssh_control_hash = hashlib.sha256(cluster_name.encode("utf-8")).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode("utf-8")).hexdigest()

gemini-code-assist · 2026-01-16T22:13:43Z

python/ray/tests/test_command_runner.py

+    ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
+    ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

ssh_control_hash = hashlib.sha256(cluster_name.encode("utf-8")).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode("utf-8")).hexdigest()

gemini-code-assist · 2026-01-16T22:13:43Z

python/ray/tests/test_command_runner.py

+    ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
+    ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

ssh_control_hash = hashlib.sha256(cluster_name.encode("utf-8")).hexdigest()

ssh_user_hash = hashlib.sha256(getuser().encode("utf-8")).hexdigest()

gemini-code-assist · 2026-01-16T22:13:43Z

python/ray/util/collective/const.py

    if not group_name:
        raise ValueError("group_name is None.")
-    hexlified_name = hashlib.sha1(group_name.encode()).hexdigest()
+    hexlified_name = hashlib.sha256(group_name.encode()).hexdigest()


For consistency and to avoid platform-dependent behavior, it's better to explicitly specify the encoding as utf-8 when calling .encode().

Suggested change

hexlified_name = hashlib.sha256(group_name.encode()).hexdigest()

hexlified_name = hashlib.sha256(group_name.encode("utf-8")).hexdigest()

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

python/ray/data/preprocessors/utils.py

Pure refactoring. The file isn't used anywhere. Unused code adds to maintenance burden. Signed-off-by: irabbani <[email protected]> Signed-off-by: Purushotham Pushpavanth <[email protected]>

This commit addresses security concerns by migrating all internal hash operations from SHA-1 to SHA-256 across the Ray codebase. Fixes ray-project#53435 Signed-off-by: Purushotham Pushpavanth <[email protected]>

## Description When the autoscaler commits to draining a node, immediately update `ClusterResourceManager` so the scheduler sees the node as unavailable. ### Why There's a race condition where work can be scheduled to nodes that are committed for draining: 1. Autoscaler calls `HandleDrainNode()` -> `SetNodeDraining()` updates `GcsNodeManager::draining_nodes_`. 2. Scheduler checks `ClusterResourceManager::IsNodeDraining()` which reads from `NodeResources::is_draining`. 3. But `is_draining` is only updated when the raylet broadcasts via RaySyncer. During this window, scheduler sees `is_draining=false` and schedules work to the draining node. To fix this, `GcsResourceManager` now registers a listener with `GcsNodeManager` that immediately updates `ClusterResourceManager` when a node is set to draining. This closes the race window where work could be scheduled to draining nodes. --------- Signed-off-by: Sagar Sumit <[email protected]> Co-authored-by: Zac Policzer <[email protected]> Signed-off-by: Purushotham Pushpavanth <[email protected]>

Signed-off-by: Purushotham Pushpavanth <[email protected]>

edoakes

The C++ changes look irrelevant here -- maybe a bad merge?

I believe some of these hashes are used in the runtime_env directory paths. I'd be concerned that extending their length could cause problems in some environments. I don't think this change in those areas is worth the risk of breakage.

This commit reverts runtime_env hash functions back to SHA-1 to address reviewer concerns about filesystem path length limitations. Runtime environment hashes are used in: - Directory paths: /tmp/ray/.../pip/<hash> - GCS URIs: gcs://_ray_pkg_<hash>.zip - Conda env names: ray-<hash> SHA-256 would increase hash length from 40 to 64 characters (+24 chars), which could cause issues on: - Windows systems (PATH_MAX = 260 characters) - Network filesystems with shorter path limits - Legacy systems with path restrictions Additionally, changing these hashes would invalidate all existing cached runtime environments, causing unnecessary rebuilds. For content-addressable hashing (non-cryptographic use), SHA-1 provides sufficient collision resistance. SHA-256 is retained for security-critical operations like SSH control paths, config hashing, and CloudWatch configs. Files reverted to SHA-1: - python/ray/_private/runtime_env/pip.py - python/ray/_private/runtime_env/uv.py - python/ray/_private/runtime_env/conda.py - python/ray/_private/runtime_env/conda_utils.py - python/ray/_private/runtime_env/packaging.py Related to ray-project#53435 Signed-off-by: Purushotham Pushpavanth <[email protected]>

pushpavanthar · 2026-01-17T16:39:31Z

@edoakes Thank you for the feedback!

The C++ changes look irrelevant here -- maybe a bad merge?

You are right about the bad merge, fixed it by pulling master again.

I believe some of these hashes are used in the runtime_env directory paths. I'd be concerned that extending their length could cause problems in some environments. I don't think this change in those areas is worth the risk of breakage.

Great catch! I've reverted the 5 runtime_env files back to SHA-1 to avoid the path length issues you identified. SHA-256 would add +24 characters to filesystem paths, which could definitely cause problems on Windows and network filesystems, plus it would invalidate all existing cached environments.

Signed-off-by: Purushotham Pushpavanth <[email protected]>

pushpavanthar requested review from a team as code owners January 16, 2026 22:02

pushpavanthar force-pushed the SHA256 branch from 5ec0c61 to 098ba19 Compare January 16, 2026 22:04

gemini-code-assist bot reviewed Jan 16, 2026

View reviewed changes

cursor bot reviewed Jan 16, 2026

View reviewed changes

python/ray/data/preprocessors/utils.py Show resolved Hide resolved

pushpavanthar force-pushed the SHA256 branch from 098ba19 to f0b64bb Compare January 16, 2026 22:16

israbbani and others added 4 commits January 17, 2026 03:54

[core] Deleting unused gcs_server_test_util.h (ray-project#60232)

9792860

Pure refactoring. The file isn't used anywhere. Unused code adds to maintenance burden. Signed-off-by: irabbani <[email protected]> Signed-off-by: Purushotham Pushpavanth <[email protected]>

[core] Replace SHA-1 with SHA-256 for internal hash operations

e590c40

This commit addresses security concerns by migrating all internal hash operations from SHA-1 to SHA-256 across the Ray codebase. Fixes ray-project#53435 Signed-off-by: Purushotham Pushpavanth <[email protected]>

Updated the test file with the correct SHA-256 expected values

1a3712a

Signed-off-by: Purushotham Pushpavanth <[email protected]>

pushpavanthar force-pushed the SHA256 branch from 6ce597b to 1a3712a Compare January 16, 2026 22:24

edoakes reviewed Jan 17, 2026

View reviewed changes

ray-gardener bot added core Issues that should be addressed in Ray Core community-contribution Contributed by the community labels Jan 17, 2026

pushpavanthar added 2 commits January 17, 2026 21:53

Merge branch 'master' of https://github.com/ray-project/ray into SHA256

799b3b9

[core] Update test expected values for SHA-256 migration

72ad997

Signed-off-by: Purushotham Pushpavanth <[email protected]>

pushpavanthar requested a review from edoakes January 17, 2026 22:53

Merge branch 'master' into SHA256

190f15b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[core] Replace SHA-1 with SHA-256 for internal hash operations #60242

[core] Replace SHA-1 with SHA-256 for internal hash operations #60242

Uh oh!

pushpavanthar commented Jan 16, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

gemini-code-assist bot Jan 16, 2026

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

edoakes left a comment

Uh oh!

pushpavanthar commented Jan 17, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	sha256.update(str(filepath.relative_to(relative_path)).encode())
	sha256.update(str(filepath.relative_to(relative_path)).encode("utf-8"))

		ssh_control_hash = hashlib.sha256(cluster_name.encode()).hexdigest()
		ssh_user_hash = hashlib.sha256(getuser().encode()).hexdigest()

	hexlified_name = hashlib.sha256(group_name.encode()).hexdigest()
	hexlified_name = hashlib.sha256(group_name.encode("utf-8")).hexdigest()

[core] Replace SHA-1 with SHA-256 for internal hash operations #60242

Are you sure you want to change the base?

[core] Replace SHA-1 with SHA-256 for internal hash operations #60242

Uh oh!

Conversation

pushpavanthar commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

SHA-256 Migration (15 files)

SHA-1 Retained (5 files)

Testing

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

edoakes left a comment

Choose a reason for hiding this comment

Uh oh!

pushpavanthar commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pushpavanthar commented Jan 16, 2026 •

edited

Loading

pushpavanthar commented Jan 17, 2026 •

edited

Loading