Invalid pod spec when deploying `resource` template definition with `mainContainer.env` configured.

[oterno]

Pre-requisites

• I have double-checked my configuration
• I can confirm the issues exists when I tested with :latest
• I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

I have installed argo-workflows with helm (chart version 0.22.11). With mainContainer.env it is possible to add environment variables to the main container in workflows. This works fine for workflows using the container template definition. However, if a workflow uses the resource template definition, the workflow will fail with an error as below:

Pod "workflow-deploy-resource-m4nbm" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: "": may not be specified when `value` is not empty

In the minimal reproducible example, I have provided two workflows doing approximately the same job; deploying a whalesay container. The configMap argo-workflows-workflow-controller-configmap must have at least one env variable defined under .config:

mainContainer:
  env:
    - name: MY_ENV_VAR
      value: example_value

NOTE: The workflow-controller deployment must be restarted after modifying the configMap.

Version

v3.4.5

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-deploy-resource-
  namespace: argo-workflows
  labels:
    workflows.argoproj.io/archive-strategy: "false"
spec:
  templates:
    - name: main
      resource:
        action: apply
        manifest: |
          apiVersion: v1
          kind: Pod
          metadata:
            name: whalesay
            namespace: argo-workflows
          spec:
            containers:
              - name: main
                image: docker/whalesay:latest
                command:
                  - cowsay
                args:
                  - hello world
            restartPolicy: Never
  entrypoint: main
  serviceAccountName: argo-workflows-workflow-controller
---
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: workflow-deploy-container-
  namespace: argo-workflows
  labels:
    workflows.argoproj.io/archive-strategy: "false"
spec:
  templates:
    - name: main
      container:
        image: docker/whalesay:latest
        command: [cowsay]
        args: ["hello world"]
  entrypoint: main
  serviceAccountName: argo-workflows-workflow-controller

Logs from the workflow controller

time="2023-02-17T14:26:00.892Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.900Z" level=info msg="Updated phase  -> Running" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.900Z" level=info msg="Pod node workflow-deploy-resource-m4nbm initialized Pending" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.910Z" level=warning msg="Non-transient error: Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty"
time="2023-02-17T14:26:00.910Z" level=info msg="Failed to create pod workflow-deploy-resource-m4nbm (workflow-deploy-resource-m4nbm): Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=warning msg="Non-transient error: Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty"
time="2023-02-17T14:26:00.911Z" level=error msg="Mark error node" error="Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty" namespace=argo-workflows nodeName=workflow-deploy-resource-m4nbm workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="node workflow-deploy-resource-m4nbm phase Pending -> Error" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="node workflow-deploy-resource-m4nbm message: Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="node workflow-deploy-resource-m4nbm finished: 2023-02-17 14:26:00.911316703 +0000 UTC" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=error msg="error in entry template execution" error="Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=warning msg="Non-transient error: Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty"
time="2023-02-17T14:26:00.911Z" level=info msg="Updated phase Running -> Error" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="Updated message  -> error in entry template execution: Pod \"workflow-deploy-resource-m4nbm\" is invalid: spec.containers[0].env[0].valueFrom: Invalid value: \"\": may not be specified when `value` is not empty" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="Marking workflow completed" namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.911Z" level=info msg="Checking daemoned children of " namespace=argo-workflows workflow=workflow-deploy-resource-m4nbm
time="2023-02-17T14:26:00.917Z" level=info msg="cleaning up pod" action=deletePod key=argo-workflows/workflow-deploy-resource-m4nbm-1340600742-agent/deletePod
time="2023-02-17T14:26:00.922Z" level=info msg="Workflow update successful" namespace=argo-workflows phase=Error resourceVersion=10438090 workflow=workflow-deploy-resource-m4nbm

Logs from in your workflow's wait container

N/A

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If this is a mentoring request, please provide an update here. Thank you for your contributions.

[beanscg]

Bean Labs took a public-source pass on this issue.

The narrowest useful next step looks like: manifest validation path

Public evidence checked:

• workflow/controller/
• docs/

A good acceptance check would be a docs diff that changes only the affected wording/link and renders without stale references.

No PR opened from this pass.