KFLUXINFRA-3695 Install Kueue v1.3 on internal-production#341
KFLUXINFRA-3695 Install Kueue v1.3 on internal-production#341manish-jangra wants to merge 3 commits into
Conversation
manish-jangra
requested review from
enkeefe00,
gbenhaim and
hugares
as code owners
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: manish-jangra The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Review Summary by QodoAdd Kueue configuration for internal-production cluster
WalkthroughsDescription• Deploy Kueue operator with CatalogSource, Subscription, and Kueue CR • Configure ClusterQueue with BestEffortFIFO and resource quotas • Set up tekton-kueue controller and webhook with HA configuration • Add LocalQueue and WorkloadPriorityClass for queue-based scheduling • Implement CEL expressions for priority assignment and rate-limiting Diagramflowchart LR
A["Kueue Operator<br/>CatalogSource v4.18"] --> B["Kueue CR<br/>Managed State"]
B --> C["ClusterQueue<br/>cluster-pipeline-queue"]
C --> D["LocalQueue<br/>pipelines-queue"]
D --> E["tekton-kueue<br/>Controller & Webhook"]
C --> F["ResourceFlavor<br/>default-flavor"]
E --> G["CEL Expressions<br/>Priority & Rate-limiting"]
File Changes1. components/kueue/internal-production/kueue/operator.yaml
|
Code Review by Qodo
Context used✅ Tickets:
🎫 Installation of Tekton-Kueue 1. config.yaml missing in base
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| resources: | ||
| - https://github.com/konflux-ci/tekton-kueue/config/default?ref=815dbf158b0e39df568eb901dcf05e759099991a | ||
|
|
There was a problem hiding this comment.
1. Tekton-kueue base component missing 📎 Requirement gap ⚙ Maintainability
The PR adds tekton-kueue manifests only under the internal-production overlay (via a remote upstream resource) but does not create the required reusable base at components/kueue/base/tekton-kueue/. This fails the checklist requirement for a pinned, shared base component derived from upstream config/default.
Agent Prompt
## Issue description
The PR does not add the required base component at `components/kueue/base/tekton-kueue/` derived from the pinned upstream `config/default` manifests; it only adds an environment-specific overlay.
## Issue Context
Compliance requires a reusable base component (pinned to a specific ref) so that staging/production overlays can share the same upstream-derived manifests while allowing overlay-specific tuning.
## Fix Focus Areas
- components/kueue/internal-production/tekton-kueue/kustomization.yaml[4-6]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
There was a problem hiding this comment.
The agreement within team is not to use shared overlay like base due to some past experiences.
|
/review |
PR Reviewer Guide 🔍Warning
Here are some key observations to aid the review process:
|
manish-jangra
changed the title
Add the kueue component overlay for the internal-production cluster, enabling queue-based scheduling for signing PipelineRuns. Configuration mirrors staging as the baseline and will be tuned independently based on production observations. Includes: - Kueue operator (CatalogSource, Subscription, Kueue CR) - ClusterQueue with BestEffortFIFO and signing-server-request resource - LocalQueue in internal-services namespace - tekton-kueue controller and webhook with HA patches - CEL expressions for priority assignment and rate-limiting Co-authored-by: Cursor <cursoragent@cursor.com>
Upgrade the Kueue operator subscription channel from stable-v1.2 to stable-v1.3 and bump all Kueue CRD apiVersions from v1beta1 to v1beta2 (ResourceFlavor, ClusterQueue, WorkloadPriorityClass, LocalQueue) to match the new API surface shipped with v1.3. Mirrors the staging upgrade in PR redhat-appstudio#342. Co-authored-by: Cursor <cursoragent@cursor.com>
…alQueue Move tekton-kueue sync wave from 10 to 12 so it deploys after the LocalQueue (wave 11). This prevents a race where the tekton-kueue webhook becomes active before the pipelines-queue LocalQueue exists, which could cause PipelineRuns to be rejected during rollout. Fixes both internal-staging and internal-production overlays. Co-authored-by: Cursor <cursoragent@cursor.com>
manish-jangra
force-pushed
the
KFLUXINFRA-3695
branch
from
6a2d579 to
6811469
Compare
|
/agentic_review |
|
Persistent review updated to latest commit 6811469 |
| configMapGenerator: | ||
| - name: tekton-kueue-config | ||
| namespace: tekton-kueue | ||
| behavior: replace | ||
| files: | ||
| - config.yaml | ||
|
|
There was a problem hiding this comment.
1. config.yaml missing in base 📎 Requirement gap ⚙ Maintainability
The PR adds config.yaml only within the environment-specific tekton-kueue directory rather than providing the required placeholder within components/kueue/base/tekton-kueue/. This prevents the base component from packaging the placeholder for future CEL rule configuration.
Agent Prompt
## Issue description
`config.yaml` exists only in the environment-specific tekton-kueue directory, but compliance requires a placeholder `config.yaml` to live inside the base tekton-kueue component and be packaged by that component's kustomization.
## Issue Context
The placeholder is required so future CEL rules can be added by updating the base component used by multiple overlays.
## Fix Focus Areas
- components/kueue/internal-production/tekton-kueue/kustomization.yaml[17-23]
- components/kueue/internal-production/tekton-kueue/config.yaml[1-13]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
2 participants
Summary
stable-v1.3channel withv1beta2CRD apiVersionsContext
Kueue has never been installed on the internal-production cluster. This PR introduces a fresh installation at v1.3.1, aligned with the staging deployment.
Related PRs
Changes
Production install (new files)
kueue/— Namespace, CatalogSource (v4.18 index), OperatorGroup, Subscription (stable-v1.3), Kueue CR with Tekton PipelineRun frameworkqueue-config/— ClusterQueue (cluster-pipeline-queue) withsigning-server-requestresource + WorkloadPriorityClass (v1beta2)localqueues/— LocalQueue (pipelines-queue) ininternal-servicesnamespace (v1beta2)tekton-kueue/— Controller (2 replicas) and webhook (3 replicas) with topology spread, resource limits, and leader election tuningSync wave fix (both environments)
tekton-kueue sync wave moved from 10 → 12 so it deploys after LocalQueue (wave 11), preventing a race where the webhook starts before
pipelines-queueexists.Sync wave ordering
kueue/queue-config/localqueues/tekton-kueue/Note
Both environments are kept independent (no shared base overlay) to allow independent tuning — per team agreement.