Skip to content

resourceQuota plugin double-calculates quotas #4220

Description

@yaoyinnan

Description

The resourceQuota plug-in has duplicate calculations, causing the check to fail and the podgroup to be in a non-Running state.

Steps to reproduce the issue

  1. After creating the following resourceQuota:
apiVersion: v1
kind: ResourceQuota
metadata:
  name: cpu-test-resource-quota
  namespace: cpu-test
spec:
  hard:
    limits.cpu: "100m"
    limits.memory: "1000"
  1. First create a pod that occupies 50m cpu and 300 memory:
apiVersion: v1
kind: Pod
metadata:
  name: test-resourcequota-pod-1
  namespace: cpu-test
  labels:
    app: test-resourcequota-pod-1
spec:
  containers:
  - name: container
    image: ubuntu
    imagePullPolicy: IfNotPresent
    command:
    - /bin/sh
    - -c
    - sleep 365d
    resources:
      limits:
        cpu: 50m
        memory: 300
    securityContext:
      runAsUser: 0
  restartPolicy: Never
  schedulerName: volcano
  1. View resourceQuota at this time:
apiVersion: v1
kind: ResourceQuota
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"ResourceQuota","metadata":{"annotations":{},"name":"cpu-test-resource-quota","namespace":"cpu-test"},"spec":{"hard":{"limits.cpu":"100m","limits.memory":"1000"}}}
  creationTimestamp: "2025-04-24T02:23:46Z"
  name: cpu-test-resource-quota
  namespace: cpu-test
  resourceVersion: "424209"
  uid: 92d0e0eb-3a99-4d9b-910d-6d6dab80c1e6
spec:
  hard:
    limits.cpu: 100m
    limits.memory: 1k
status:
  hard:
    limits.cpu: 100m
    limits.memory: 1k
  used:
    limits.cpu: 50m
    limits.memory: "300"
  1. Create another pod that takes up 40m of CPU and 400 of memory:
apiVersion: v1
kind: Pod
metadata:
  name: test-resourcequota-pod-2
  namespace: cpu-test
  labels:
    app: test-resourcequota-pod-2
spec:
  containers:
  - name: container
    image: ubuntu
    imagePullPolicy: IfNotPresent
    command:
    - /bin/sh
    - -c
    - sleep 365d
    resources:
      limits:
        cpu: 40m
        memory: 400
    securityContext:
      runAsUser: 0
  restartPolicy: Never
  schedulerName: volcano

As expected, the resourceQuota limit should not be reached, but scheduling is not successful.

  1. Checking the podgroup of the Pod created later shows that the resource quota is insufficient.
Name:         podgroup-6dcfb3e7-7be4-4bc6-9af0-0320c73bc24d
Namespace:    cpu-test
Labels:       <none>
Annotations:  <none>
API Version:  scheduling.volcano.sh/v1beta1
Kind:         PodGroup
Metadata:
  Creation Timestamp:  2025-04-24T03:14:50Z
  Generation:          3
  Managed Fields:
    API Version:  scheduling.volcano.sh/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
          .:
          k:{"uid":"6dcfb3e7-7be4-4bc6-9af0-0320c73bc24d"}:
      f:spec:
        .:
        f:minMember:
        f:minResources:
          .:
          f:count/pods:
          f:cpu:
          f:limits.cpu:
          f:limits.memory:
          f:memory:
          f:pods:
          f:requests.cpu:
          f:requests.memory:
      f:status:
        .:
        f:phase:
    Manager:      volcano-controller-manager
    Operation:    Update
    Time:         2025-04-24T03:14:50Z
    API Version:  scheduling.volcano.sh/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
    Manager:    volcano-scheduler
    Operation:  Update
    Time:       2025-04-24T03:14:51Z
  Owner References:
    API Version:           v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Pod
    Name:                  test-resourcequota-pod-2
    UID:                   6dcfb3e7-7be4-4bc6-9af0-0320c73bc24d
  Resource Version:        427151
  UID:                     9cd20e06-4ed2-4dd0-b2a3-11dc7392ff90
Spec:
  Min Member:  1
  Min Resources:
    count/pods:       1
    Cpu:              40m
    limits.cpu:       40m
    limits.memory:    400
    Memory:           400
    Pods:             1
    requests.cpu:     40m
    requests.memory:  400
  Queue:              default
Status:
  Conditions:
    Last Transition Time:  2025-04-24T03:14:51Z
    Message:               1/1 tasks in gang unschedulable: pod group is not ready, 1 Pending, 1 minAvailable; Pending: 1 Unschedulable
    Reason:                NotEnoughResources
    Status:                True
    Transition ID:         ca80b61d-7004-4813-a69f-192a41eb7c9c
    Type:                  Unschedulable
  Phase:                   Pending
Events:
  Type     Reason         Age                 From     Message
  ----     ------         ----                ----     -------
  Warning  Unschedulable  41s                 volcano  0/0 tasks in gang unschedulable: pod group is not ready, 1 minAvailable
  Warning  Unschedulable  30s (x11 over 40s)  volcano  1/1 tasks in gang unschedulable: pod group is not ready, 1 Pending, 1 minAvailable; Pending: 1 Unschedulable
  Normal   Unschedulable  29s (x13 over 41s)  volcano  resource quota insufficient, requested: map[limits.cpu:{{40 -3} {<nil>} 40m DecimalSI} limits.memory:{{400 0} {<nil>} 400 DecimalSI}], used: map[limits.cpu:{{90 -3} {<nil>} 90m DecimalSI} limits.memory:{{700 0} {<nil>} 700 DecimalSI}], limited: map[limits.cpu:{{100 -3} {<nil>} 100m DecimalSI} limits.memory:{{1 3} {<nil>} 1k DecimalSI}]

Because after obtaining the Used of resourceQuota, job.PodGroup.Spec.MinResources is accumulated, but the value of Used itself has already added the resource request of the created Pod.

Describe the results you received and expected

As expected, the resourceQuota limit should not be reached, but scheduling is not successful.

What version of Volcano are you using?

v1.10.0

Any other relevant information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions