Inconsistent requeue strategy for missing dependency-like references

[matheuscscp]

Describe the bug

When a controller is waiting for a dependency-like resource to appear (source object, valuesFrom ConfigMap/Secret, substituteFrom ConfigMap/Secret), the requeue strategy is inconsistent with how dependsOn and "source exists but not ready" are handled.

Both helm-controller and kustomize-controller use --requeue-dependency for dependsOn not-ready and source-exists-but-artifact-missing, but use different (worse) strategies for other "waiting for a resource" scenarios.

helm-controller

Scenario	Requeue strategy
dependsOn not ready	--requeue-dependency
Source exists but not ready	--requeue-dependency
Artifact fetch file not found	--requeue-dependency
Source object not found (chartRef)	exponential backoff
valuesFrom not found	exponential backoff

Source object not found (helmrelease_controller.go#L288-L290):

msg := fmt.Sprintf("could not get Source object: %s", err.Error())
conditions.MarkFalse(obj, meta.ReadyCondition, v2.ArtifactFailedReason, "%s", msg)
return ctrl.Result{}, err

valuesFrom not found (helmrelease_controller.go#L318-L320):

conditions.MarkFalse(obj, meta.ReadyCondition, "ValuesError", "%s", err)
r.Eventf(obj, corev1.EventTypeWarning, "ValuesError", err.Error())
return ctrl.Result{}, err

kustomize-controller

Scenario	Requeue strategy
dependsOn not ready	--requeue-dependency
Source artifact nil	--requeue-dependency
Artifact fetch file not found	--requeue-dependency
Source object not found	retryInterval
substituteFrom not found	retryInterval

Source object not found (kustomization_controller.go#L214-L217):

if apierrors.IsNotFound(err) {
    msg := fmt.Sprintf("Source '%s' not found", obj.Spec.SourceRef.String())
    log.Info(msg)
    return ctrl.Result{RequeueAfter: obj.GetRetryInterval()}, nil
}

substituteFrom not found — caught as a general reconcile error (kustomization_controller.go#L289-L297):

if reconcileErr != nil {
    log.Error(reconcileErr, fmt.Sprintf("Reconciliation failed after %s, next try in %s",
        time.Since(reconcileStart).String(),
        obj.GetRetryInterval().String()),
        "revision",
        revision)
    r.event(obj, revision, originRevision, eventv1.EventSeverityError,
        reconcileErr.Error(), nil)
    return ctrl.Result{RequeueAfter: obj.GetRetryInterval()}, nil
}

Expected behavior

All "waiting for a dependency-like resource to exist" scenarios should requeue with --requeue-dependency, consistent with dependsOn not-ready and source-not-ready handling.

Use case

This matters when resources are deployed together in the same sync path (e.g. via a ResourceSet). A HelmRelease may reference a valuesFrom ConfigMap generated by a Kustomize configMapGenerator, or a chartRef OCIRepository — these may not exist yet when the HelmRelease is first reconciled. Exponential backoff (or the full retry/reconcile interval) delays the eventual reconciliation unnecessarily compared to the short --requeue-dependency interval.

[Iam-Karan-Suresh]

Hi @matheuscscp, I’d like to work on this issue if it’s still available. Could you please assign it to me?

[matheuscscp]

Let's see if other maintainers agree this needs fixing or not

[texasich]

Hi @matheuscscp — happy to pick this up if maintainer alignment lands on "yes". I'd plan two PRs (one against helm-controller, one against kustomize-controller), routing the source-not-found / valuesFrom / substituteFrom branches to requeueDependency() (or whatever the equivalent on each reconciler is) so they share the short-interval path with dependsOn.

Holding off on a draft until you confirm it should be fixed. If it gets aligned in the other direction (status-quo intentional), no worries — happy to close out.

[vecil]

Thanks for raising this 🙌 it would improve predictability and make Flux easier to reason about in both development and production.

Since this has been open for a while, it’d be great if a fix could be considered.

[matheuscscp]

@vecil Are you facing any undesired behavior related to this issue?

[vecil]

@vecil Are you facing any undesired behavior related to this issue?

I’m not seeing any undesired behavior on my side, so it’s not a blocker.
My comment was wrong, I meant improving consistency and responsiveness, not reporting an issue.

[Bisman-Singh]

I'd like to take this one. Plan per the issue's pointers: align the not-found paths (source object, valuesFrom, substituteFrom) in helm-controller and kustomize-controller with the --requeue-dependency interval already used for not-ready dependencies, with test coverage for each referenced kind. I'll open PRs in the two controller repos referencing this issue.

[Bisman-Singh]

PRs are up: fluxcd/helm-controller#1577 (source not found + non-optional valuesFrom not found) and fluxcd/kustomize-controller#1734 (source not found + substituteFrom not found), both requeuing at the --requeue-dependency interval, mirroring the existing not-ready idiom, with envtest coverage per path. Opened as drafts pending the maintainer consensus mentioned above — happy to adjust the approach or mark ready whenever that lands.

[matheuscscp]

Not sure the fix is changing the remaining things to --requeue-dependency anymore. What is retryInterval for, then? Exponential backoff is def. worth fixing, but not sure about moving things from retryInterval to --requeue-dependency.

[Bisman-Singh]

Good question, and the answer turns out to be different for each controller.

kustomize-controller (#1734): you are right, and there is no exponential backoff to fix there. Every reconcile failure already ends at return ctrl.Result{RequeueAfter: obj.GetRetryInterval()}, nil, and the source-not-found branch already used obj.GetRetryInterval() as well. So both hunks in that PR move a user-tunable retryInterval onto the global --requeue-dependency flag, which is exactly what you are questioning. I am happy to close it, or to reduce it to only the missing substituteFrom case if you consider a not-yet-created reference a dependency rather than a failure. Your call.

helm-controller (#1577) is a different situation. Both paths I changed returned a bare error (return ctrl.Result{}, err), so they went to controller-runtime's rate limiter rather than to any interval a user can set. retryInterval in the HelmRelease API is spec.install.retryInterval and spec.upgrade.retryInterval, which retry a failed Helm install or upgrade and are applied further down through obj.GetActiveRetry(...).GetRetryInterval(). Nothing in that PR displaces them. The change makes a missing source object and a missing non-optional valuesFrom reference behave like the not-ready dependency branch just above them, which already carries the comment "Exponential backoff would cause execution to be prolonged too much, instead we requeue on a fixed interval".

So the split I would suggest is to keep #1577 as the exponential-backoff fix you agree is worth making, and drop or shrink #1734. Tell me which and I will update both.