Using Kustomize to add/update DAG-tasks to a workflow in different envs.

[eiriklid]

I have the use-case where I want to add some tasks to test and run them a little while before I update prod with the same task.

In my kustomization.yaml I have:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../resources/argo-workflows

...

patches:
  - path: argo-workflows-patch.yaml
    target:
      kind: WorkflowTemplate
      name: workflow-1

workflow-template-1.yaml:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-1
spec:
entrypoint: run-workflow-1
templates:
    - name:  run-workflow-1
      dag:
        tasks:
          - name: A
            templateRef:
              name: job-templates
              template: redacted

          - name: B
            templateRef:
              name: job-templates
              template: redacted

argo-workflows-patch.yaml:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-1
spec:
  templates:
    - name: run-workflow-1
      dag:
        tasks:
          - name: C
            dependencies:
              - A
              - B
            templateRef:
              name: job-templates
              template: redacted

          - name: D
            dependencies:
              - A
            templateRef:
              name: job-templates
              template: redacted

When I run kubectl kustomize manifests/overlay/dev/ it is only the content from argo-workflows-patch.yaml that shows up and not task A and B.

From https://github.com/argoproj/argo-workflows/blob/main/api/openapi-spec/swagger.json#L11981 and other definitions in the same file, it looks like it should be able to merge, so that task A, B, C and D shows up in run-workflow-1.

Do anyone see what I'm doing wrong?

I have tried many different approaches to patching with Kustomize, but the only one I have been able to successfully implement is this:

- patch: |

    - op: add
      path: /spec/templates/0/dag/tasks/- # Add a new task to the workflow
      value: 
        name: C
        dependencies:
          - A
          - B
        templateRef:
          name: job-templates
          template: redacted
          

which is ok, but I also ned to change dependencies for some task,

- op: replace
        path: /spec/templates/0/dag/tasks/8/dependencies # Add a new dependency to the H task
        value:
          - E
    target:
      kind: WorkflowTemplate
      name: workflow-curves-with-dependencies

which can become quite confusing quickly...

[static-moonlight]

Do you have a resources/argo-workflows/kustomization.yaml which includes workflow-template-1.yaml, ... ?

Otherwise you could try to include them directly in manifests/overlay/dev/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../resources/argo-workflows/workflow-template-1.yaml
  - ../../resources/argo-workflows/workflow-template-2.yaml
  - ../../resources/argo-workflows/workflow-template-3.yaml

...

patches:
  - path: argo-workflows-patch.yaml
    target:
      kind: WorkflowTemplate
      name: workflow-1