Dynamically Provision Manifests with AppSet if a directory exists

[deepbrook]

Given the following the directory structure:

apps/
  common/
    additionalResources.appSet.yaml
  control-plane/  # Management cluster
    my-app/
      <manifests> ... # Manifests to be provisioned across all 'control-plane' clusters
  environments/
    my-app/
      <manifests> ...  # Manifests to be provisioned across all 'environment' clusters
      dev/
         <manifests> ...  # Manifests to be provisioned to the dev cluster **only**

Let's assume the following clusters exist:

• control-plane
• environments-dev
• environments-test

I've already rolled out the core part of my-app, using an applicationSet and the cluster generator. Now, I want to have the ability to dynamically provision additional resources for my-app (for example, additional secrets, config maps, etc).

I want to be able to do this as follows:

• manifests in <cluster-type>/my-app should be provisioned to all clusters of the given type (control-plane or environments suffix)
• manifests in <cluster-type>/my-app/<env> should be provisioned to all clusters of the given type (control-plane or environments prefix) and env name (dev or test suffix, in this example).
• if both exist, both should be provisioned

I'm currently using two ApplicationSets for this, with the following generators:

For cluster-type specific resources:

spec:
  generators:
    - matrix:
        generators:
          - list:
              elements:  # using a list generator, as i will add additional apps in the future
                - my-app
          - clusters:
              values:
                cluster_type: '{{ .metadata.labels["cluster-type"]}}'
                cluster_env: '{{ .metadata.labels["env"]}}'
          - scm:
              gitlab:
                group: my-group
              filters:
                - repositoryMatch: ^my-k8s-repo  # specifically match the repo our config lives in
                  pathsExist:
                    - "apps/{{ .values.cluster_type }}/{{ .app_name }}"
              values:
                resource_path: "apps/{{ .values.cluster_type }}/{{ .app_name }}"

And for env-specific resources, I have the exact same setup but use apps/{{ .values.cluster_type }}/{{ .app_name }}/{{ .values.cluster_env }} as filter on the scm generator:

spec:
  generators:
    - matrix:
        generators:
          - list:
              elements:  # using a list generator, as i will add additional apps in the future
                - my-app
          - clusters:
              values:
                cluster_type: '{{ .metadata.labels["cluster-type"]}}'
                cluster_env: '{{ .metadata.labels["env"]}}'
          - scm:
              gitlab:
                group: my-group
              filters:
                - repositoryMatch: ^my-k8s-repo  # specifically match the repo our config lives in
                  pathsExist:
                    - "apps/{{ .values.cluster_type }}/{{ .app_name }}/{{ .values.cluster_env }}"
              values:
                resource_path: "apps/{{ .values.cluster_type }}/{{ .app_name }}/{{ .values.cluster_env }}"

Side note: I'm not using the git generator with directories, because I want to keep the ability to have arbitrary sub-directories.

If i used the following git generator:

- git:
    repoUrl: "..."
    directories:
    - path: "apps/{{ values.cluster_type }}"  # <- lists directories outside of the `list` generator's `elements`
    - path: "apps/{{ values.cluster_type }}/{{ .app_name }}" # <- Lists directories which potentially are not in named after a valid env 

I would have a whole lot of undesired values in the matrix generator.

So, to sum up: my problem is that manifests may or may not exist at a path, and depending on that I want provision those manifests (or not).

Is there any way to do this in a single applicationSet, or do I have to either live with several ones or need to give up arbitrary sub-dirs?

[kostis-codefresh]

Please check this detailed guide of mine https://codefresh.io/blog/argocd-clusters-labels-with-apps/

And associated Git repo https://github.com/kostis-codefresh/multi-app-multi-value-argocd

TL;DR you need to use multiple application sets