Is it possible to stop reconciliation of a HelmRelease before timeout?

[ilya-git]

I have a helm chart with a deployment.
I have created a HelmRelease manifest that contains an error that makes a pod in this deployment to never be created (stuck in container creating). The error is not related to flux and can probably be any other kind of similar error.

I have now committed the fix, but it is not applied until flux generates this event:
Helm upgrade failed: timed out waiting for the condition

If I understand things correctly Flux is waiting 5 minutes to declare the release failed and then it takes the latest version and does a new rollout that succeeds.

What I am wondering is if there is a way to force Flux to abandon the ongoing reconciliation (so that it can take latest version of release) and not wait for the timeout? I was not able to find such command.

[hiddeco]

The timed out waiting for the condition comes from Helm.

What the controller does equals to using the --wait flag while running helm install or helm upgrade. Exiting this process early is not possible as the controller has no idea what Helm is waiting for or what specific error it times out on, that's something only you, as a human and user, are able to observe based on detecting the error and triaging what goes wrong.

It is possible to disable the --wait behavior by configuring disableWait. In most cases you do however not want to do this, as this may result in errors going unnoticed.

[ilya-git]

Thanks a lot for the answer! The best thing for me to do if I don't want to wait (at least while I am actively developing and there are a lot of errors) is to reduce the default timeout on HelmRelease from 5 minutes to e.g. 1 minute?

[hiddeco]

That is an option, but it would likely be better to simply confirm your chart is working correctly using helm install [--wait], as this gives you direct feedback (and gives you the option to cancel things instantly by simply pressing Ctrl + C). Once your chart is working as expected, you can then start automating it in a HelmRelease.

Another reason to do this is because configuring the timeout to a lower value may result in unexpected side-effects, if your chart for example has e.g. jobs attached to it that take a longer time to complete, the timeout may be too low.

[ilya-git]

Thanks for tips, I was relying so much of flux, I did not think of the easiest solution :)

[wmgroot]

I'd love if Flux supported an option to cancel an ongoing helm install or helm upgrade in response to a new revision of the HelmRelease being pushed to the K8s API. This would not be usable for all deployments, since some might require a hard wait for migrations or other work to be performed prior to a new upgrade starting.

But for a large number of deployment shapes, canceling an in-progress rollout to start rolling out the newest version is desirable behavior. One use case is minimizing remediation time while rolling out reverts or other fixes. With the current behavior, you need to wait for an in-progress failing rollout to time out and fail before you can attempt to roll out the fix.

[matheuscscp]

This will be implemented here:

#5584

[wmgroot]

Awesome!

[makkes]

Well, to be precise, #5584 is for Kustomizations whereas for Helm we would depend on the feature being implemented in Helm 4.

[matheuscscp]

Feature being implemented in Helm here:

helm/helm#31435

[matheuscscp]

It's merged, it will land in Helm v4.0.0. Flux will release Helm 4 support in 2.8, early next year.