[Core] Add fallback strategy support to placement groups by ryanaoleary · Pull Request #59024 · ray-project/ray

ryanaoleary · 2025-11-27T02:05:51Z

Description

This PR adds a fallback_strategy argument to placement groups. For Ray placement groups, fallback_strategy is a list of scheduling option dicts, where the arguments can be bundles and (optionally) bundle_label_selector, to try in order until one can be scheduled. This PR also updates the GCS scheduler to consider these fallbacks when calling ScheduleUnplacedBundles. If a fallback strategy is chosen, the bundles in the placement group table data will be updated to the selected value.

Edit: We've updated the proto fields where bundles now only shows the actively chosen bundles for scheduling, and scheduling_strategy contains the list of all scheduling options (primary bundles and fallback bundles).

Related issues

#51564

ryanaoleary · 2025-11-27T02:05:59Z

cc: @MengjinYan

gemini-code-assist

Code Review

This pull request introduces a fallback_strategy for placement groups, allowing for alternative scheduling options if the primary configuration is not feasible. The changes are well-implemented across the Python API, Cython layer, and C++ core, including updates to the GCS scheduler. The addition of comprehensive unit and integration tests ensures the new functionality is robust. The code is well-structured, and I've identified one minor opportunity for code simplification to enhance maintainability.

src/ray/gcs/gcs_placement_group.cc

python/ray/util/placement_group.py

github-actions · 2025-12-11T12:26:02Z

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

MengjinYan · 2025-12-11T18:29:37Z

not stale

MengjinYan

Some high level comments. Will take a more detailed look after those are resolved. Thanks!

python/ray/util/placement_group.py

python/ray/tests/test_placement_group_fallback.py

src/ray/gcs/gcs_placement_group.cc

src/ray/protobuf/common.proto

MengjinYan

Thanks for the patience! Added one high level comment.

src/ray/gcs/gcs_placement_group.cc

src/ray/common/placement_group.h

src/ray/protobuf/gcs.proto

src/ray/protobuf/common.proto

python/ray/util/placement_group.py

MengjinYan · 2026-01-14T23:38:40Z

src/ray/gcs/gcs_placement_group.h

By the new semantics of the bundles field, I think the field should be empty in the beginning. And probably same with the strategy if we add it to the placement group scheduling options

Tried adding this and it runs into a lot of issues with the python client and the autoscaler, since they expect bundles to be populated for the pending placement group. To fix it we end up having to modify how bundles get cached and essentially return scheduling_strategy index 0 (the primary strategy) anytime bundles would get called anyway.

The change I tried to implement ended up being pretty expansive, is this something we should try to add in a follow-up PR? Or do you know if I might be missing something and there's an easier way to support the new semantics of the field.

Understood. I think it is a very involved change because the semantic changes.

The current placement group management logic in all places assumes that there is only one set of requirements for each placement group so the requirements and the state tracking are all in the bundles field.

At the same time, with the fallback semantic, while the user facing API remains the bundles will still be used for specifying the highest priority resource requirement, we changed the assumption in GCS placement group management logic and split the original bundles field in PlacementGroupTableData into:

scheduling_strategy which only contains the scheduling requirements for the bundles

and bundles which only tracks the bundle id and the corresponding node id for the currently chosen placement group. Only bundle_id and node_id needs to be populated and we might need an additional field to indicate the index of the chosen set of requirements in the scheduling_strategy

Looking into the code, I think on a high level, the semantic changes means:

The user facing APIs should remain the same in terms of the bundles field and just add fallback_options additional field:

semantics in user facing APIs (placement_group.py) shouldn't change

semantics in PlacementGroupSpecification, BundleSpecification etc. shouldn't change

On GCS side, the semantics in GcsPlacementGroup should change to the new semantics. This means:
(1) The operations related bundle field (GetBundles, GetUnplacedBundles, etc.) should be changed to the semantics of assuming this is the chosen resource requirements. cached_bundle_specs_ should be the chosen
(2) The placement group scheduling logic needs to consider all the resource requirements in scheduling_strategy when the whole placement group needs to be scheduled.
(3) The logic to sync the pending placement groups from GCS to autoscaler needs to provide the list of resource requirements

In terms of the PRs, looks like the current PR handles (2) and related changes to update the PlacementGroupTableData and PlacementGroupSpec data model. And if the current didn't change the existing behavior of having one set of requirements in bundles, I think we can add the additional changes that covers (1) and (3) in separate PRs.

Let me know whether the above makes sense or I missed some aspects in handling the bundles semantic changes.

cc: @edoakes for visibility and if you have additional comment.

Makes sense, thanks for outlining it I feel like that made it a lot more clear to implement. I went through and think with af673db I've implemented everything from (2) and part of (1). I changed bundles to remain empty until an active strategy has been selected, but updated GetBundles to return the highest priority strategy when bundles is empty so that it still works with pending PGs. I think a follow-up PR would still be needed so that this field fully follow the new semantics and returns an empty list before a strategy has been chosen (and also updates all downstream consumers to check for

For the autoscaler, I updated GetPendingGangResourceRequests since it reads from the proto to check for scheduling_strategy(0) for PENDING groups, and then in a follow-up PR that addresses the autoscaling side can have it actually consider the full list of fallback options when scheduling.

src/ray/protobuf/gcs.proto

src/ray/gcs/gcs_placement_group_scheduler.cc

python/ray/util/placement_group.py

src/ray/protobuf/gcs.proto

python/ray/tests/test_placement_group_fallback.py

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

src/ray/gcs/gcs_autoscaler_state_manager.cc

src/ray/protobuf/common.proto

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

python/ray/util/placement_group.py

src/ray/gcs/gcs_placement_group.h

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

python/ray/_private/state.py

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

src/ray/protobuf/common.proto

python/ray/util/placement_group.py

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

ryanaoleary · 2026-02-07T02:42:56Z

cc: @MengjinYan I had a bad rebase that ended up dropping some of my changes to the files, I've now added them back in and re-tested. The semantics of the fields is now being followed per #59024 (comment).

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

python/ray/util/placement_group.py

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

python/ray/util/placement_group.py

python/ray/tests/test_placement_group_fallback.py

cursor · 2026-02-07T07:55:23Z

src/ray/protobuf/common.proto

  // Label selector associated with this bundle.
  // Populated from bundle_label_selector if provided.
-  map<string, string> label_selector = 4;
+  LabelSelector label_selector = 4;


Proto files modified without RPC fault-tolerance review

Low Severity

⚠️ This PR modifies one or more .proto files.
Please review the RPC fault-tolerance & idempotency standards guide here:
https://github.com/ray-project/ray/tree/master/doc/source/ray-core/internals/rpc-fault-tolerance.rst

Additional Locations (1)

src/ray/protobuf/gcs.proto#L686-L687

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

cursor · 2026-02-07T12:44:36Z

src/ray/protobuf/common.proto

  // Label selector associated with this bundle.
  // Populated from bundle_label_selector if provided.
-  map<string, string> label_selector = 4;
+  LabelSelector label_selector = 4;


Proto files modified require RPC fault-tolerance review

Low Severity

⚠️ This PR modifies one or more .proto files.
Please review the RPC fault-tolerance & idempotency standards guide here:
https://github.com/ray-project/ray/tree/master/doc/source/ray-core/internals/rpc-fault-tolerance.rst

Additional Locations (1)

src/ray/protobuf/gcs.proto#L686-L687

cursor · 2026-02-07T12:44:36Z

src/ray/gcs/gcs_placement_group.h

+    // Index 1..N: fallback strategies.
+    placement_group_table_data_.mutable_scheduling_options()->MergeFrom(
+        placement_group_spec.fallback_strategy());
+


Empty bundles field breaks bundle index validation

High Severity

The GcsPlacementGroup constructor no longer populates the bundles field (it only populates scheduling_options). This means bundles is empty for all newly created placement groups until the scheduler calls UpdateActiveBundles. Since HandleGetPlacementGroup serves from in-memory state, client-side bundle_count returns 0 before scheduling runs. This causes check_placement_group_index to reject any placement_group_bundle_index >= 0 with a ValueError. The race window can be significant when other PGs are scheduling concurrently (IsSchedulingInProgress blocks the queue).

Additional Locations (1)

python/ray/util/placement_group.py#L132-L140

ryanaoleary requested a review from a team as a code owner November 27, 2025 02:05

gemini-code-assist bot reviewed Nov 27, 2025

View reviewed changes

cursor bot reviewed Nov 27, 2025

View reviewed changes

src/ray/gcs/gcs_placement_group.cc Outdated Show resolved Hide resolved

python/ray/util/placement_group.py Show resolved Hide resolved

ray-gardener bot added core Issues that should be addressed in Ray Core community-contribution Contributed by the community labels Nov 27, 2025

github-actions bot added the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Dec 11, 2025

MengjinYan self-assigned this Dec 11, 2025

MengjinYan removed the stale The issue is stale. It will be closed within 7 days unless there are further conversation label Dec 11, 2025

MengjinYan reviewed Dec 12, 2025

View reviewed changes

python/ray/util/placement_group.py Show resolved Hide resolved

python/ray/tests/test_placement_group_fallback.py Show resolved Hide resolved

src/ray/gcs/gcs_placement_group.cc Outdated Show resolved Hide resolved

src/ray/gcs/gcs_placement_group.cc Outdated Show resolved Hide resolved

ryanaoleary mentioned this pull request Dec 18, 2025

[Serve] Add label_selector and bundle_label_selector to Serve API #57694

Merged

8 tasks

ryanaoleary requested a review from MengjinYan December 18, 2025 19:49

cursor bot reviewed Dec 18, 2025

View reviewed changes

src/ray/protobuf/common.proto Show resolved Hide resolved

MengjinYan reviewed Jan 10, 2026

View reviewed changes

src/ray/gcs/gcs_placement_group.cc Outdated Show resolved Hide resolved

ryanaoleary requested a review from MengjinYan January 14, 2026 05:21

cursor bot reviewed Jan 14, 2026

View reviewed changes

src/ray/gcs/gcs_placement_group.cc Show resolved Hide resolved

src/ray/common/placement_group.h Outdated Show resolved Hide resolved

MengjinYan reviewed Jan 15, 2026

View reviewed changes

cursor bot reviewed Jan 22, 2026

View reviewed changes

python/ray/util/placement_group.py Outdated Show resolved Hide resolved

ryanaoleary requested a review from MengjinYan January 22, 2026 20:27

MengjinYan mentioned this pull request Jan 26, 2026

[Core] Ray Label Selector API Implementation Tracker #51564

Open

35 tasks

ryanaoleary force-pushed the pg-fallback-strategy branch from af673db to 5650ee9 Compare February 3, 2026 23:00

ryanaoleary requested review from a team, matthewdeng and richardliaw as code owners February 3, 2026 23:00

aslonnie removed request for a team, SongGuyang, WangTaoTheTonic, aslonnie, kfstorm and raulchen February 4, 2026 00:58

Merge branch 'master' into pg-fallback-strategy

893dee4

cursor bot reviewed Feb 5, 2026

View reviewed changes

src/ray/protobuf/gcs.proto Outdated Show resolved Hide resolved

python/ray/tests/test_placement_group_fallback.py Show resolved Hide resolved

ryanaoleary and others added 2 commits February 6, 2026 11:55

Add back changes dropped in bad rebase

cc7a8a4

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

Merge branch 'master' into pg-fallback-strategy

b42757b

cursor bot reviewed Feb 6, 2026

View reviewed changes

src/ray/gcs/gcs_autoscaler_state_manager.cc Show resolved Hide resolved

src/ray/protobuf/common.proto Show resolved Hide resolved

Fix loop to iterate over bundles_source

f69cbaa

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor bot reviewed Feb 6, 2026

View reviewed changes

python/ray/util/placement_group.py Show resolved Hide resolved

src/ray/gcs/gcs_placement_group.h Show resolved Hide resolved

Fix cursor comments and rename field to be clear its not strategy

fcefe96

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor bot reviewed Feb 6, 2026

View reviewed changes

python/ray/_private/state.py Outdated Show resolved Hide resolved

python/ray/_private/state.py Show resolved Hide resolved

Fix erroneously named field

2306900

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor bot reviewed Feb 7, 2026

View reviewed changes

src/ray/protobuf/common.proto Show resolved Hide resolved

python/ray/util/placement_group.py Outdated Show resolved Hide resolved

python/ray/util/placement_group.py Outdated Show resolved Hide resolved

ryanaoleary and others added 3 commits February 7, 2026 02:39

Fix cacheing in placement group

3a74c66

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

Clang format

27e4a6e

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

Merge branch 'master' into pg-fallback-strategy

0c5a113

cursor bot reviewed Feb 7, 2026

View reviewed changes

python/ray/util/placement_group.py Show resolved Hide resolved

ryanaoleary and others added 2 commits February 7, 2026 07:42

Fix truthiness of cache check

8f6bbcb

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

Merge branch 'master' into pg-fallback-strategy

972ad3c

cursor bot reviewed Feb 7, 2026

View reviewed changes

Fix cache call and comment

060e03d

Signed-off-by: ryanaoleary <ryanaoleary@google.com>

cursor bot reviewed Feb 7, 2026

View reviewed changes

Conversation

ryanaoleary commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues

Uh oh!

ryanaoleary commented Nov 27, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Dec 11, 2025

Uh oh!

MengjinYan commented Dec 11, 2025

Uh oh!

MengjinYan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MengjinYan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MengjinYan Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

ryanaoleary Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

MengjinYan Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

ryanaoleary Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ryanaoleary commented Feb 7, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ryanaoleary commented Nov 27, 2025 •

edited

Loading