Skip to content

[FLINK-33548] Replace taskManager/jobmanager.resource with resources conforming to ResourceRequirements#1082

Open
vsantwana wants to merge 3 commits intoapache:mainfrom
vsantwana:FLINK-33548
Open

[FLINK-33548] Replace taskManager/jobmanager.resource with resources conforming to ResourceRequirements#1082
vsantwana wants to merge 3 commits intoapache:mainfrom
vsantwana:FLINK-33548

Conversation

@vsantwana
Copy link
Copy Markdown
Contributor

@vsantwana vsantwana commented Apr 2, 2026

What is the purpose of the change

This pull request introduces a new resources field on JobManagerSpec and TaskManagerSpec that conforms to the standard Kubernetes ResourceRequirements schema (with requests and limits), replacing the non-standard resource field which only supported a flat cpu/memory/ephemeralStorage structure. The old resource field is deprecated but remains fully functional as a fallback.

Brief change log

  • Added resources field (Kubernetes ResourceRequirements) to JobManagerSpec and TaskManagerSpec, deprecated the old resource field
  • Updated FlinkConfigBuilder to prioritize resources over resource in applyJobManagerSpec(), applyTaskManagerSpec(), and applyPodTemplate()
  • Added setResourceRequirements() method that maps requests.memory and requests.cpu to Flink config, and computes CPU_LIMIT_FACTOR and MEMORY_LIMIT_FACTOR from the
    limits/requests ratio
  • Added configureCpuFromString() to parse Kubernetes CPU quantity strings (e.g. "500m", "1.5")
  • Updated DefaultValidator to validate ResourceRequirements fields and accept memory defined via either resource.memory or resources.requests.memory
  • Updated CRD schema for flinkdeployments to include the new resources field
  • Fixed FlinkBlueGreenDeploymentControllerTest to use builder pattern instead of all-args constructor

Verifying this change

(Please pick either of the following options)

This change added tests and can be verified as follows:

  • Added testApplyJobManagerSpecWithResourceRequirements — verifies JM memory and CPU are set from ResourceRequirements
  • Added testApplyTaskManagerSpecWithCpuResourceRequirements — verifies milliCPU parsing ("500m" -> 0.5)
  • Added testResourceRequirementsCpuLimitFactor — verifies CPU limit factor is computed as limits.cpu / requests.cpu
  • Added testResourceRequirementsMemoryLimitFactor — verifies memory limit factor is computed as limits.memory / requests.memory
  • Added testResourceRequirementsTakesPrecedenceOverResource — verifies resources wins when both resource and resources are set
  • Added testApplyResourceRequirementsToPodTemplate — verifies ResourceRequirements are correctly merged into pod template containers
  • Added testDeprecatedResourceStillWorksWhenResourcesIsNull — verifies backward compatibility of the old resource field
  • Added testTaskManagerSpecWith2_GiResourceRequirementsSetting — verifies Kubernetes memory format "2Gi" is correctly parsed
  • Added DefaultValidator tests verifying ResourceRequirements with valid memory, valid memory+CPU, and resources-only (no deprecated resource) all pass validation

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changes to the CustomResourceDescriptors: yes
  • Core observer or reconciler logic that is regularly executed: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? Javadocs

@vsantwana vsantwana marked this pull request as ready for review April 2, 2026 16:36
properties:
deploymentName:
type: string
type: "string"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to check why this change and if it is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant