Skip to content

Pod YAML templates: spec.resources (pod-level) is dropped during pod assembly #2842

Description

@edgarkz

What feature do you want to see added?

Summary

Pod-level spec.resources defined in pipeline or pod-template YAML is not present on the Pod created by the Kubernetes plugin. Other PodSpec fields from the same YAML (for example affinity, tolerations, containers) are applied correctly.

This is both:

  1. Bug — YAML fields documented as supported are silently discarded.
  2. Feature request — Support Kubernetes pod-level resource limits/requests (Pod-level resources), which are useful for multi-container CI pods where per-container limits are hard to tune and sum incorrectly for scheduling.

Related: JENKINS-64787 / #2428 (YAML merge does not honor all PodSpec fields). spec.resources appears to be another missing field.


Use case

We run heavy multi-container Jenkins agents (main workload container, Docker sidecar, test runner, plugin-injected agent container, and optional sidecars) on single-node instance types. We want one pod-level resource budget matching node allocatable capacity instead of duplicating inflated limits on every container.

Example goal:

spec:
  resources:
    limits:
      cpu: "4"
      memory: 28Gi

We intentionally omit per-container limits so containers share the pod budget (Kubernetes PodLevelResources feature).


Steps to reproduce

  1. Configure a pipeline agent with raw pod YAML containing spec.resources and yamlMergeStrategy merge():
pipeline {
  agent {
    kubernetes {
      cloud '<kubernetes-cloud-name>'
      namespace '<namespace>'
      defaultContainer '<main-container-name>'
      yaml '''
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: example-agent
spec:
  resources:
    limits:
      cpu: "4"
      memory: 28Gi
      ephemeral-storage: 60Gi
  tolerations:
  - key: <taint-key>
    operator: Equal
    value: "true"
    effect: NoSchedule
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node.kubernetes.io/instance-type
            operator: In
            values:
            - <instance-type>
  containers:
  - name: <main-container-name>
    image: <registry>/<image>:<tag>
    command: ["sleep"]
    args: ["99d"]
  - name: <sidecar-container-name>
    image: <registry>/<image>:<tag>
    command: ["<entrypoint>"]
'''
      yamlMergeStrategy merge()
    }
  }
  stages {
    stage('Check pod') {
      steps {
        sh 'sleep 30'
      }
    }
  }
}
  1. Run the pipeline.
  2. Inspect the created pod:
kubectl get pod -n <namespace> -l <agent-label-selector> -o yaml

Expected behavior

The created pod spec includes:

spec:
  resources:
    limits:
      cpu: "4"
      memory: 28Gi

Actual behavior

spec.resources is missing entirely. Only plugin-injected agent container requests appear (default when no limits are configured on the agent container):

spec:
  containers:
  - name: <plugin-agent-container>   # typically "jnlp"
    resources:
      requests:
        cpu: 100m
        memory: 256Mi
  # affinity, tolerations, and custom containers from YAML are present
  # spec.resources is absent

Analysis

Pod assembly in PodTemplateBuilder.build():

Pod pod = combine(template.getYamlsPod(), builder.endSpec().build());

PodTemplateUtils.combine(Pod parent, Pod template) explicitly merges only a subset of PodSpec fields (containers, volumes, tolerations, nodeSelector, serviceAccount, etc.). There is no handling of PodSpec.resources:

Fields such as affinity from YAML survive because they are copied via withNewSpecLike(parent.getSpec()). Pod-level resources does not appear in the rendered pod, which suggests either:

  1. The field is not deserialized from YAML into the fabric8 PodSpec model used by the plugin, and/or
  2. The field is not preserved/merged when combining YAML with the plugin-built pod.

Container-level resources do work — they are merged in combine(Container, Container) via .withNewResources().


Proposed fix

  1. Parse spec.resources from pod YAML (ensure fabric8 / kubernetes-client model includes PodSpec.resources, or preserve unknown fields).
  2. Merge pod-level resources in PodTemplateUtils.combine(Pod, Pod) — e.g. template YAML wins over parent when set, similar to serviceAccountName.
  3. Serialize spec.resources when submitting the pod to the API.
  4. Add regression tests:
    • YAML with only spec.resources (no container resources)
    • YAML with both pod-level and container-level resources
    • merge() and override() strategies
  5. Document supported pod-level resource types (cpu, memory, hugepages) and note that ephemeral-storage remains container-level per Kubernetes API.

References


Upstream changes


Are you interested in contributing this feature?


Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    No fields configured for Enhancement.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions