Skip to content

Add kueue component #6358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

gbenhaim
Copy link
Member

Install kueue and the plugins for supporting tekton pipelineruns and external admission.
Install only as part of the development overlay and only on OCP cluster with version >= 4.16 (the kueue operator doesn't work on earlier version).

ATM, the index image for the RH productized version of Kueue isn't yet ready, because of that it's being installed using the operator-framework cli.

@gbenhaim gbenhaim requested review from hugares, sadlerap and filariow May 11, 2025 08:23
@openshift-ci openshift-ci bot requested review from elsony and johnmcollier May 11, 2025 08:23
Copy link

openshift-ci bot commented May 11, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gbenhaim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gbenhaim gbenhaim force-pushed the add-kueue-component branch from b487ab0 to 15622d6 Compare May 11, 2025 08:59
@gbenhaim
Copy link
Member Author

/test konflux-e2e-v416-optional

@gbenhaim
Copy link
Member Author

/test

Copy link

openshift-ci bot commented May 12, 2025

@gbenhaim: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test appstudio-e2e-tests
/test appstudio-load-test

The following commands are available to trigger optional jobs:

/test appstudio-hac-e2e-tests
/test appstudio-upgrade-tests
/test konflux-e2e-v416-optional

Use /test all to run the following jobs that were automatically triggered:

pull-ci-redhat-appstudio-infra-deployments-main-appstudio-e2e-tests
pull-ci-redhat-appstudio-infra-deployments-main-appstudio-upgrade-tests

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@gbenhaim
Copy link
Member Author

/test appstudio-e2e-tests
/test appstudio-upgrade-tests
/test konflux-e2e-v416-optional

Copy link
Contributor

@sadlerap sadlerap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions and comments, but mostly looks fine by me.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm failing to understand why we need cert-manager. Could you maybe provide some clarity on why openshift's certificate management capabilities isn't used?

I'm hoping we can continue to keep certificate management controlled by openshift on our clusters, not split between it and cert-manager.

kind: Namespace
metadata:
name: openshift-kueue-operator
spec: {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
spec: {}

roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: admin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to grant this service account admin rights on the namespace?

Comment on lines +112 to +119
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "200m"
memory: "256Mi"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set the requests and the limits to be the same to avoid pod evictions?

@gbenhaim gbenhaim force-pushed the add-kueue-component branch from 448b7d4 to dad9ff1 Compare May 13, 2025 06:36
@gbenhaim
Copy link
Member Author

/test konflux-e2e-v416-optional

gbenhaim added 11 commits May 14, 2025 10:35
The kueue extensions will enable the use of Kueue with Tekton
and controlling workload admission based on control plane alerts.

Signed-off-by: Gal Ben Haim <[email protected]>
It's required for running Kueue and its extensions.

Signed-off-by: Gal Ben Haim <[email protected]>
- Add the manifests for deploying Kueue using the RH operator.
  Since wasn't released yet (and it doesn't have an index image),
  use operator-sdk for installing it.

- Add the ArgoCD AppSet for Kueue.

Signed-off-by: Gal Ben Haim <[email protected]>
The configmap is used for telling the admission controller if it
need to admit workloads. This is a temporary solution until the
integration with alert manager will by implemented.

Signed-off-by: Gal Ben Haim <[email protected]>
It's not possible to install it on earlier versions.

Signed-off-by: Gal Ben Haim <[email protected]>
Signed-off-by: Gal Ben Haim <[email protected]>
@gbenhaim
Copy link
Member Author

/test konflux-e2e-v416-optional

@gbenhaim
Copy link
Member Author

/test konflux-e2e-v416-optional

Copy link

openshift-ci bot commented May 14, 2025

@gbenhaim: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/appstudio-e2e-tests b653014 link true /test appstudio-e2e-tests
ci/prow/konflux-e2e-v416-optional b653014 link false /test konflux-e2e-v416-optional

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-robot
Copy link
Collaborator

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants