[Data] Cleaning up `ResourceManager` and `ReservationOpResourceAllocator` by alexeykudinkin · Pull Request #60273 · ray-project/ray

alexeykudinkin · 2026-01-18T18:09:15Z

Description

Cleaning up ResourceManager

Cleaning up methods duplication
Fixing _should_unblock_streaming_output_backpressure semantic
Abstracting common _is_blocking_materializing_op util to determine if operation is a blocking materializing op

Cleaning up ReservationOpResourceAllocator

Adjusting can_submit_new_task to check for available Object Store when launching tasks

Related issues

Link related issues: "Fixes #1234", "Closes #1234", or "Related to #1234".

Additional information

Optional: Add implementation details, API changes, usage examples, screenshots, etc.

gemini-code-assist

Code Review

This pull request provides a solid cleanup of the ResourceManager and related components. The changes clarify the handling of materializing operators, refactor memory usage accounting, and centralize logic for completed operators. The removal of object store memory estimation when no samples are available simplifies the code. The test updates are also comprehensive and improve test quality by using more realistic mocks.

I have a couple of suggestions to improve code clarity by avoiding shadowed variables in list/generator comprehensions.

python/ray/data/_internal/execution/resource_manager.py

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Revisiting selection of eligible ops Signed-off-by: Alexey Kudinkin <ak@anyscale.com> # Conflicts: # python/ray/data/_internal/execution/resource_manager.py Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

…bytes` Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Updated usages Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

cursor · 2026-01-30T22:09:19Z

python/ray/data/_internal/execution/resource_manager.py

            logger.warning(msg)

-    def __init__(self, topology: "Topology"):
-        self._topology = topology


Missing rate limiting update in idle detection

Medium Severity

The detect_idle method no longer updates last_detection_time[op] when detecting idle state. Previously, this timestamp was updated in both branches (active and idle). Without this update, after the first idle detection, every subsequent call will pass the interval check because last_detection_time remains stale, breaking the rate-limiting behavior. This causes print_warning_if_idle_for_too_long to be called repeatedly and _should_unblock_streaming_output_backpressure to always return True for idle operators.

This is incorrect. I've added a test covering expected behavior for IdleDetector

python/ray/data/_internal/execution/streaming_executor_state.py

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

…tor` (ray-project#60273) ## Description Cleaning up `ResourceManager` - Cleaning up methods duplication - Fixing `_should_unblock_streaming_output_backpressure` semantic - Abstracting common `_is_blocking_materializing_op` util to determine if operation is a blocking materializing op Cleaning up `ReservationOpResourceAllocator` - Adjusting `can_submit_new_task` to check for available Object Store when launching tasks ## Related issues > Link related issues: "Fixes ray-project#1234", "Closes ray-project#1234", or "Related to ray-project#1234". ## Additional information > Optional: Add implementation details, API changes, usage examples, screenshots, etc. --------- Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: 400Ping <jiekaichang@apache.org>

…tor` (ray-project#60273) ## Description Cleaning up `ResourceManager` - Cleaning up methods duplication - Fixing `_should_unblock_streaming_output_backpressure` semantic - Abstracting common `_is_blocking_materializing_op` util to determine if operation is a blocking materializing op Cleaning up `ReservationOpResourceAllocator` - Adjusting `can_submit_new_task` to check for available Object Store when launching tasks ## Related issues > Link related issues: "Fixes ray-project#1234", "Closes ray-project#1234", or "Related to ray-project#1234". ## Additional information > Optional: Add implementation details, API changes, usage examples, screenshots, etc. --------- Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Signed-off-by: Sirui Huang <ray.huang@anyscale.com>

alexeykudinkin requested a review from a team as a code owner January 18, 2026 18:09

alexeykudinkin force-pushed the ak/rm-div-rec branch from aa9553e to 1fd6e80 Compare January 18, 2026 18:10

gemini-code-assist bot reviewed Jan 18, 2026

View reviewed changes

python/ray/data/_internal/execution/resource_manager.py Show resolved Hide resolved

python/ray/data/_internal/execution/resource_manager.py Show resolved Hide resolved

ray-gardener bot added the data Ray Data-related issues label Jan 18, 2026

alexeykudinkin added regression go add ONLY when ready to merge, run all tests and removed regression labels Jan 20, 2026

alexeykudinkin force-pushed the ak/rm-div-rec branch from 1fd6e80 to 2b1bca1 Compare January 23, 2026 00:48

This comment was marked as outdated.

Sign in to view

cursor bot reviewed Jan 23, 2026

View reviewed changes

python/ray/data/_internal/execution/resource_manager.py Show resolved Hide resolved

alexeykudinkin force-pushed the ak/rm-div-rec branch from 98d7bf0 to 88d9709 Compare January 30, 2026 19:44

cursor bot reviewed Jan 30, 2026

View reviewed changes

python/ray/data/_internal/execution/resource_manager.py Show resolved Hide resolved

alexeykudinkin added 18 commits January 30, 2026 13:58

Simplified obj_store_mem_max_pending_output_per_task

a8339ff

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Make ATAO & Zip ineligible for resource allocation

5158b41

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Remove OS from op task requirements

c344f31

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Revisit OS usage estimation

ad90efa

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Revisited handling of blocking materializing ops

185586b

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Tidying up RM;

d9cc655

Revisiting selection of eligible ops Signed-off-by: Alexey Kudinkin <ak@anyscale.com> # Conflicts: # python/ray/data/_internal/execution/resource_manager.py Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Abstracted common utilizatio subtraction

63516a6

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Make sure usage accounts for ineligible ops' usage

89aac4a

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Rebased usages onto _is_blocking_materializing_op

a208700

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Fixed tests

5088446

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Updated tests

314cc42

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Updated RM tests

5ec0d39

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Updated metrics tests

ead94fa

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Updated RBRA and RM tests

81090ca

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Missing

8af0114

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Fixed test to be more predictable

f9e4fd0

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

lint

7d079e8

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Added total_enqueued_output_blocks & `total_enqueued_output_blocks_…

78aaad0

…bytes` Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

alexeykudinkin added 5 commits January 30, 2026 13:58

Revisiting _should_unblock_streaming_output_backpressure

8fc18c8

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Fixed tests

327c608

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Streamlining ResourceManager API;

43837b9

Updated usages Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Fixed tests

ed7ebd0

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

Added checking for Object Store memory into can_submit_new_task

f8bae90

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

alexeykudinkin force-pushed the ak/rm-div-rec branch from d060cf5 to f8bae90 Compare January 30, 2026 21:58

alexeykudinkin changed the title ~~[WIP][Data] Cleaning up ResourceManager and ReservationOpResourceAllocator~~ [Data] Cleaning up ResourceManager and ReservationOpResourceAllocator Jan 30, 2026

cursor bot reviewed Jan 30, 2026

View reviewed changes

lint

4bdd2a5

Signed-off-by: Alexey Kudinkin <ak@anyscale.com>

bveeramani approved these changes Jan 30, 2026

View reviewed changes

alexeykudinkin enabled auto-merge (squash) January 30, 2026 23:28

alexeykudinkin merged commit b570dc2 into master Jan 31, 2026
7 checks passed

alexeykudinkin deleted the ak/rm-div-rec branch January 31, 2026 00:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Data] Cleaning up `ResourceManager` and `ReservationOpResourceAllocator`#60273

[Data] Cleaning up `ResourceManager` and `ReservationOpResourceAllocator`#60273
alexeykudinkin merged 24 commits intomasterfrom
ak/rm-div-rec

alexeykudinkin commented Jan 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Uh oh!

cursor bot Jan 30, 2026

Uh oh!

alexeykudinkin Jan 30, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

alexeykudinkin commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues

Additional information

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Jan 30, 2026

Choose a reason for hiding this comment

Missing rate limiting update in idle detection

Uh oh!

alexeykudinkin Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alexeykudinkin commented Jan 18, 2026 •

edited

Loading