Skip to content

Pass label selector predicate to the Secret Watch and handler #443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

kurlov
Copy link
Contributor

@kurlov kurlov commented Mar 27, 2025

What

Updated the setupWatches method to pass predicate if labelSelector is set to the reconciler.

Why

If run two reconcilers with the same GVK but different labelSelectors then both reconcilers reconcile each other RCs ignoring configured LabelSelectors. This could be considered as not expected behavior if running two or more reconcilers with the same GVK but different labelSelectors is covered scenario. This PR fixes this issue.

Example logs of two reconcilers with the same GVK but different label selector.
There is only one CR with label app=nginx.
Reconciler1 has app=nginx label selector
Reconciler2 has app=foo label selector

Reconciler1:


2025-03-27T02:58:33+01:00       INFO    controllers.Helm        Watching resource       {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T02:58:33+01:00       INFO    setup   starting manager with label selector    {"selector": "app=nginx"}
2025-03-27T02:58:33+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T02:58:33+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T02:58:33+01:00       INFO    Starting Controller     {"controller": "nginx-controller"}
2025-03-27T02:58:33+01:00       INFO    Starting workers        {"controller": "nginx-controller", "worker count": 1}
2025-03-27T02:58:33+01:00       DEBUG   controllers.Helm        Reconciliation triggered        {"nginx": {"name":"nginx-sample","namespace":"default"}}
...

Reconciler2:

2025-03-27T03:00:39+01:00       INFO    controllers.Helm        Watching resource       {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:00:39+01:00       INFO    setup   starting manager with label selector    {"selector": "app=foo"}
2025-03-27T03:00:39+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:00:39+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:00:39+01:00       INFO    Starting Controller     {"controller": "nginx-controller"}
2025-03-27T03:00:39+01:00       INFO    Starting workers        {"controller": "nginx-controller", "worker count": 1}
2025-03-27T03:00:39+01:00       DEBUG   controllers.Helm        Reconciliation triggered        {"nginx": {"name":"nginx-sample","namespace":"default"}}

Manual testing

This small setup was used for running locally two reconcilers:

Running locally with the fix shows that only one reconciler reconcile the CR and another

Reconciler1:

2025-03-27T03:18:53+01:00       INFO    controllers.Helm        Watching resource       {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:18:53+01:00       INFO    setup   starting manager with label selector    {"selector": "app=nginx"}
2025-03-27T03:18:53+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:18:53+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:18:53+01:00       INFO    Starting Controller     {"controller": "nginx-controller"}
2025-03-27T03:18:53+01:00       INFO    Starting workers        {"controller": "nginx-controller", "worker count": 1}
2025-03-27T03:18:53+01:00       DEBUG   controllers.Helm        Reconciliation triggered        {"nginx": {"name":"nginx-sample","namespace":"default"}}

Reconciler2:

2025-03-27T03:18:37+01:00       INFO    controllers.Helm        Watching resource       {"group": "demo.example.com", "version": "v1alpha1", "kind": "Nginx"}
2025-03-27T03:18:37+01:00       INFO    setup   starting manager with label selector    {"selector": "app=foo"}
2025-03-27T03:18:37+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *unstructured.Unstructured"}
2025-03-27T03:18:37+01:00       INFO    Starting EventSource    {"controller": "nginx-controller", "source": "kind source: *v1.Secret"}
2025-03-27T03:18:37+01:00       INFO    Starting Controller     {"controller": "nginx-controller"}
2025-03-27T03:18:37+01:00       INFO    Starting workers        {"controller": "nginx-controller", "worker count": 1}

@@ -1028,8 +1025,18 @@ func (r *Reconciler) setupWatches(mgr ctrl.Manager, c controller.Controller) err
obj.SetGroupVersionKind(*r.gvk)

var preds []predicate.Predicate
if r.selectorPredicate != nil {
preds = append(preds, r.selectorPredicate)
var secretPreds []predicate.TypedPredicate[*corev1.Secret]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that you are harmonizing with the existing code, but do we need to use a slice at all here (for either preds or secretPreds)?

Copy link
Contributor

@perdasilva perdasilva May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, got it. It's to handle the nil case...

@@ -1060,6 +1067,7 @@ func (r *Reconciler) setupWatches(mgr ctrl.Manager, c controller.Controller) err
obj,
handler.OnlyControllerOwner(),
),
secretPreds...,
Copy link
Contributor

@perdasilva perdasilva May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we get away with fewer changes if we used client.Object in the watch, like e.g.:

if err := c.Watch(
		source.Kind(
			mgr.GetCache(),
			client.Object(secret),
			handler.TypedEnqueueRequestForOwner[client.Object](
				mgr.GetScheme(),
				mgr.GetRESTMapper(),
				obj,
				handler.OnlyControllerOwner(),
			),
			preds...,
		),
	); err != nil {
		return err
	}

then we don't need to duplicate the predicate slice

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this is better

@@ -1488,6 +1492,45 @@ var _ = Describe("Reconciler", func() {
})
})
})
When("label selector set", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

Copy link
Contributor

@perdasilva perdasilva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - just added a small suggestion that could obviate the need to refactor the option because of the requirement for a TypedPredicate. I don't know if there could be side-effects though.

@kurlov kurlov requested a review from perdasilva May 22, 2025 15:03
@perdasilva perdasilva enabled auto-merge June 13, 2025 11:20
auto-merge was automatically disabled June 16, 2025 09:10

Head branch was pushed to by a user without write access

@kurlov kurlov requested a review from perdasilva June 23, 2025 09:32
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.

Project coverage is 78.49%. Comparing base (08ab7fb) to head (c3f3347).
Report is 94 commits behind head on main.

Files with missing lines Patch % Lines
pkg/reconciler/reconciler.go 45.45% 5 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (08ab7fb) and HEAD (c3f3347). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (08ab7fb) HEAD (c3f3347)
2 1
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #443      +/-   ##
==========================================
- Coverage   85.06%   78.49%   -6.58%     
==========================================
  Files          19       31      +12     
  Lines        1346     2520    +1174     
==========================================
+ Hits         1145     1978     +833     
- Misses        125      452     +327     
- Partials       76       90      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@perdasilva perdasilva added this pull request to the merge queue Jun 30, 2025
Merged via the queue into operator-framework:main with commit 71298f1 Jun 30, 2025
6 checks passed
if r.selectorPredicate != nil {
preds = append(preds, r.selectorPredicate)

if r.labelSelector.MatchLabels != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, so AFAICT WithSelector accepts a selector with match expressions (and no match labels), but then this check causes these expressions to be ignored? 🤔
This would be an unwelcome surprise for the user, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch!
I think it depends on how operator handles selectors. If operator uses built-in ParseToLabelSelector from k8s like in stackrox then this line will not be ignored and behaviour should be correct. But if some operator which uses this plugin creates selector manually with providing MatchExpressions only then there will be still a bug for multiple version for the same operator.

I also tested fir PR locally and my setup had such selector generation and it worked as expected:

...
selector := "app=" + appLabelValue
labelSelector, err := metav1.ParseToLabelSelector(selector)
...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it depends on what you pass to ParseToLabelSelector. If you use any other operator than = or == then AFAICT MatchLabels will be empty and MatchExpressions will not.
I think the best way would be to change the if to do the emptiness check differently...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants