Refine conditions for reconciles from cluster updates across shards#4730
Merged
Conversation
Filtering all incoming events in the bundle reconciler would prevent updates to relevant clusters, which may live in shards different to the bundle's, from impacting the bundle. Instead, all bundle controllers, regardless of their shards, may now receive updates from clusters which are relevant to the bundle, considering, as was already the case, the bundle and the cluster's respective namespaces, cluster groups, and bundle namespace mappings.
A bundle deployment is created by Fleet, and bears the same shard ID as its originating bundle, as the sharding label is propagated along with all Fleet labels from a bundle to its bundle deployments. Therefore, filtering incoming bundle deployment-related events for bundle reconciles by shard ID is safe, and preserves the performance gain brought by the previous wider-reaching event filter.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refines shard-related reconcile triggering so that cluster create/update events can fan out reconciles to shard-scoped resources (Bundles and Schedules) even when the triggering Cluster lives in a different shard.
Changes:
- Adjust Bundle controller watchers/predicates so cluster events can trigger reconciles while still only reconciling Bundles/BundleDeployments owned by the current shard.
- Update Schedule controller to allow all Cluster events to trigger schedule mapping, and expand cluster→schedule mapping to include schedules that don’t yet have a scheduled job.
- Add integration tests covering schedule behavior across shard IDs and extend bundle status tests to cover cluster changes across shard IDs.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/cmd/controller/reconciler/schedule_controller.go | Refines shard filtering placement and expands cluster→schedule mapping to include non-job-backed schedules. |
| internal/cmd/controller/reconciler/bundle_controller.go | Moves shard filtering to specific watchers (Bundle/BundleDeployment) while allowing cluster events to fan out appropriately. |
| integrationtests/controller/schedule/suite_test.go | Adds an envtest-based integration suite for schedule controller behavior. |
| integrationtests/controller/schedule/schedule_test.go | Adds integration coverage for scheduling behavior when clusters change/match targets across shard IDs. |
| integrationtests/controller/bundle/status_test.go | Expands bundle status tests to ensure cluster changes (including cross-shard) trigger expected updates. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
When a cluster is created or, enabling it to newly match a schedule's targets, the schedule would not be reconciled as a result, because logic mapping clusters to schedules only considered schedules for which jobs had already been scheduled. That logic now takes all schedules into account within the cluster's namespace.
A cluster being updated would not make much of a difference, because mapping schedules to and from clusters works based on the schedules and clusters' respective namespaces. However, creating a new cluster in the same namespace still needs to be taken into account across shards, as a cluster living in a different shard may be relevant to a schedule, living in its same namespace and matching its targets.
weyfonk
force-pushed
the
4567-fix-sharded-triggers
branch
from
ebfd4be to
110de44
Compare
While the Schedule reconciler must be able to process cluster events coming from any shard, only Schedules matching the reconciler's shard must be reconciled as a result, to prevent a given Schedule from being reconciled concurrently across multiple controllers.
thardeck
approved these changes
Mar 13, 2026
weyfonk
added a commit
to weyfonk/fleet
that referenced
this pull request
Mar 13, 2026
…ancher#4730) * Enable cluster-to-bundle updates across shards Filtering all incoming events in the bundle reconciler would prevent updates to relevant clusters, which may live in shards different to the bundle's, from impacting the bundle. Instead, all bundle controllers, regardless of their shards, may now receive updates from clusters which are relevant to the bundle, considering, as was already the case, the bundle and the cluster's respective namespaces, cluster groups, and bundle namespace mappings. * Keep shard filter on bundle reconciles from deployments A bundle deployment is created by Fleet, and bears the same shard ID as its originating bundle, as the sharding label is propagated along with all Fleet labels from a bundle to its bundle deployments. Therefore, filtering incoming bundle deployment-related events for bundle reconciles by shard ID is safe, and preserves the performance gain brought by the previous wider-reaching event filter. * Enable cluster updates to trigger schedule reconciles earlier When a cluster is created or, enabling it to newly match a schedule's targets, the schedule would not be reconciled as a result, because logic mapping clusters to schedules only considered schedules for which jobs had already been scheduled. That logic now takes all schedules into account within the cluster's namespace. * Enable schedule reconciles from clusters in different shards A cluster being updated would not make much of a difference, because mapping schedules to and from clusters works based on the schedules and clusters' respective namespaces. However, creating a new cluster in the same namespace still needs to be taken into account across shards, as a cluster living in a different shard may be relevant to a schedule, living in its same namespace and matching its targets. * Prevent multiple reconciles per schedule per cluster event While the Schedule reconciler must be able to process cluster events coming from any shard, only Schedules matching the reconciler's shard must be reconciled as a result, to prevent a given Schedule from being reconciled concurrently across multiple controllers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a cluster is updated or created, bundles and schedules living in shards different to the cluster's own shard may need to be updated accordingly. Use cases include:
Incidentally, this also fixes a scheduling issue where only schedules with already scheduled jobs were considered when mapping a cluster to schedules. This prevented an update to a cluster, where the cluster newly matched a schedule's targets, from triggering a schedule reconcile.
In other words, a cluster which was updated to match a schedule's targets would not be immediately scheduled.
Refers to #4484.
Extends #4567 and may supersede it.
Additional Information
Checklist
- [ ] I have updated the documentation via a pull request in thefleet-docs repository.