Feat: Support Kubernetes DRA in karmada-scheduler-estimator

[GitHubxsy]

What would you like to be added:

The karmada-scheduler-estimator component should be enhanced to support the Kubernetes Dynamic Resource Allocation (DRA) feature. This will ensure that the estimator can accurately account for DRA-managed resources when assessing cluster resource availability.

Why is this needed:

Currently, the karmada-scheduler-estimator lacks support for Kubernetes DRA. When a Pod that depends on DRA resources is submitted, the estimator cannot correctly evaluate the resource status of the member clusters. This can lead to scheduling failures, as the scheduler may attempt to place the Pod on a cluster that cannot fulfill the DRA request.

By adding DRA support to the estimator, Karmada will be able to:

• Accurately assess the available DRA resources in each member cluster.
• Make more intelligent scheduling decisions for Pods that require dynamic resources.
• Avoid scheduling failures related to DRA resource claims.
• Improve overall resource utilization and scheduling reliability in a multi-cluster environment.

This enhancement is critical for supporting modern, resource-intensive workloads that rely on dynamic resource allocation across a federated Kubernetes environment managed by Karmada.

[RainbowMango]

Thanks for raising this feature request, @GitHubxsy!

I believe this feature is valuable for supporting modern workloads that rely on Dynamic Resource Allocation across multi-cluster environments.

To help us better understand the scope and identify the specific work items needed, could you please provide more detailed scenario descriptions? It would be great if you could share:

A concrete example: For instance, a sample Pod manifest that uses DRA resources (like ResourceClaim, ResourceClaimTemplate, etc.) and how you expect the scheduler-estimator to handle it.

Expected behavior: What specific DRA-related information should the estimator consider when assessing cluster resource availability? (e.g., ResourceSlice, DeviceClass, available devices count, etc.)

With these details, we can better define the specific tasks and design the implementation approach.

[GitHubxsy]

You're right, providing more detail will definitely help clarify the scope. Here’s a more detailed breakdown with a concrete example and the expected behavior for the scheduler-estimator.

A Concrete Example

Let's consider a scenario where a user wants to deploy a Pod that requires a GPU resource managed by DRA.

First, a ResourceClass would exist on the member cluster to define the GPU resource:

# 1. ResourceClass on a member cluster (e.g., member1)
apiVersion: resource.k8s.io/v1alpha2
kind: ResourceClass
name: gpu.example.com
driverName: gpu.example.com
# suitableForPod is true by default, meaning claims can be created for specific pods.

Next, here is a sample Pod manifest that requests a GPU using a ResourceClaimTemplate. This is a common pattern for workloads like StatefulSets or Jobs where each Pod needs its own unique resource.

# 2. Pod manifest submitted to Karmada
apiVersion: v1
kind: Pod
metadata:
  name: dra-gpu-pod
spec:
  containers:
    - name: my-gpu-app
      image: "my-app:1.0"
      command: ["/bin/sh", "-c", "sleep 3600"]
  resourceClaims:
    - name: gpu # The name of this claim within the Pod
      source:
        resourceClaimTemplate:
          metadata:
            name: dra-gpu-pod-gpu-template # A template to create the ResourceClaim
          spec:
            resourceClassName: gpu.example.com
            parametersRef:
              apiGroup: gpu.example.com
              kind: GpuParameters
              name: gpu-params-a

In this example, the Pod dra-gpu-pod requests a resource from the gpu.example.com class. The Kubernetes control plane (on the member cluster) would then dynamically create a ResourceClaim based on the template and try to find an available GPU.

Expected Behavior for the Scheduler-Estimator

When the karmada-scheduler uses the karmada-scheduler-estimator to evaluate the fitness of a member cluster (e.g., member1) for this dra-gpu-pod, the estimator should perform the following logic:

1. Identify DRA Request: The estimator must first recognize that the Pod spec contains a resourceClaims field.

2. Query DRA-related Information: To assess availability, the estimator needs to look at the state of DRA objects on the member cluster. The most critical object for this is ResourceSlice.

3. Evaluate ResourceSlice Availability:

   • The estimator should identify the ResourceClassName from the Pod's resourceClaims (gpu.example.com in our example).
   • It should then list all ResourceSlice objects on the member cluster that belong to this ResourceClass.
   • It needs to inspect these ResourceSlice objects to determine if there are any available (unallocated) devices. A ResourceSlice might look something like this:

   # A ResourceSlice on a member cluster, representing available GPUs
   apiVersion: resource.k8s.io/v1alpha2
   kind: ResourceSlice
   name: gpu-example-com-slice-a
   driverName: gpu.example.com
   nodeName: worker-node-1
   # The 'for' field indicates which ResourceClaim has been allocated this slice.
   # If a slice is available, it might not have a 'for' field, or it would
   # be clear from its status that it is ready for allocation.

   • The estimator should count how many devices are available by checking the ResourceSlices that are not currently allocated to any ResourceClaim.

4. Make a Prediction:

   • If the estimator finds at least one available and suitable device in the ResourceSlices for the gpu.example.com class, it should report to the scheduler that the member cluster has sufficient capacity.
   • If all ResourceSlices for that class are already allocated (i.e., they have a for field pointing to an existing ResourceClaim), the estimator should report that the cluster cannot satisfy the Pod's resource request.

In summary, the key change is to make the karmada-scheduler-estimator aware of ResourceClass and ResourceSlice objects. It needs to use the information within ResourceSlices to accurately predict whether a DRA-based ResourceClaim can be fulfilled on a given member cluster.

Hopefully, this provides the clarity needed to scope out the implementation.

[RainbowMango]

Thank you @GitHubxsy for providing detailed information and suggestions!

This is a valuable feature request. Supporting DRA resource scheduling in Karmada's multi-cluster environment is indeed important. I believe this feature will be a strong candidate for the project's roadmap in 2026.

We recommend the following next steps:

• Present this idea at the Karmada community meeting
  • Gather feedback and suggestions from community members
  • Determine the feature's priority and implementation approach
  • Roadmap Planning
• Prepare a comprehensive Design Proposal

We welcome you or any interested contributors to submit a design proposal document. We can further discuss this in our community meetings.

[GitHubxsy]

Thanks for the positive feedback and for outlining the next steps, @RainbowMango .

I’m very excited to hear that this feature is considered a valuable candidate for the 2026 roadmap. I completely agree that discussing it in the community meeting and preparing a design proposal are the right next steps.

Unfortunately, due to my current project commitments, I don't think I will have the capacity to prepare a comprehensive design proposal or lead the presentation at the community meeting at this time. My expertise is more focused on the use cases for DRA, and I'm not familiar enough with the internal architecture of the Karmada scheduler to write a detailed technical proposal.

However, I strongly believe in the importance of this feature for the community. I raised this issue hoping it might inspire a contributor with more experience in the scheduler components to pick it up.

I am more than happy to support anyone who takes the lead by providing detailed user scenarios, offering feedback on proposals, and helping with testing once an implementation becomes available.

Thank you again for your guidance and for considering this request. I look forward to seeing this feature move forward with the help of the community.