Configure kustomize build options

[ron1]

Kustomize has several build options listed here. These options should at least be configurable in the kustomize section of fleet.yaml. It might also be useful to allow configuration of global defaults for these values as was implemented in ArgoCD here. Note that FluxCD also allows these options to be configured here. Users migrating to Fleet from ArgoCD or FluxCD will expect these Kustomize build options to be configurable as they are in their current CD platform.

The problematic Fleet source code that hardcodes these Kustomize build options is located here. This is a blocker for our migration from ArgoCD to Fleet.

[ron1]

@StrongMonkey @nickgerace I am working on a patch that adds property kustomize.buildOptions to fleet.yaml to support kustomize build options similar to FluxCD and ArgoCD. Using the latest kustomize/v4.0.1 release, a patch that leverages package "sigs.k8s.io/kustomize/kustomize/v4/commands/build" can be implemented quite easily. However, kustomize/v4.0.x is quite new and is a breaking change from kustomize/v3.8.2, the version currently used by fleet.

In order to implement this feature using kustomize/v3.8.x, it would be most convenient to fork kustomize in order to make a few function visibility and signature changes in build.go that make it behave a bit more like v4.

Since kustomize/v4 is already released and is expected to mature quickly, I was hoping an interim, rancher-hosted kustomize fork might be an option to hold fleet over while we wait for fleet to upgrade. Do you have thoughts regarding how I should proceed?

[nickgerace]

Hi @ron1 - first of all, I really appreciate your frequent contributions to fleet and related repositories. Thank you very much.

Onto your question: I would be fine with testing with your personal work for the time being (I assume it'll be a Go module called github.com/ron1/rancher-kustomize or something similar). Once the PR looks like we are ready to merge, we may want to move it to one of our orgs: rancher or rancher-sandbox. I believe we can wait until that point though.

As a side note: are the kustomize build options in FluxCD and ArgoCD based on kustomize forks or upstream, from your knowledge?

CC: @kinarashah

[ron1]

Hi @nickgerace - I believe both FluxCD and ArgoCD use the os/exec package to invoke kustomize build. As a result, these tools are immune to the kustomize build api limitations in kustomize/v3.8.x.

Fleet OTOH uses package "sigs.k8s.io/kustomize/api/krusty" to run kustomize build. In order for Fleet to parse and validate build options/flags, it makes sense to use the kustomize "build" package. Unfortunately, the Build command in kustomize/v3.8.x does not give Fleet the control it needs to invoke kustomize build. As I mentioned above, this is fixed in kustomize/v4.0.0+.

Before I create a PR for Fleet with a forked kustomize/v3.8.x repo, I was hoping to confirm with Fleet maintainers that hosting a kustomize fork to support this feature is acceptable.

As a side note: do Fleet maintainers have a roadmap for kustomize version upgrades? Specifically, do you know when Fleet might move to kustomize/v3.8.10 to gain useful fixes over the past 6 months and when might Fleet consider a breaking move to kustomize/v4?

Since ArgoCD and FluxCD execute kustomize via a binary, users of these tools can use standard Docker techniques to specify select kustomize versions. Since Fleet uses apis to invoke kustomize, it makes sense for Fleet to stay current with the latest patch releases of tools like Helm and Kustomize.

[kkaempf]

too old