Skip to content

WIP: AI Conformance testing#17993

Open
justinsb wants to merge 11 commits intokubernetes:masterfrom
justinsb:aiconform_integration
Open

WIP: AI Conformance testing#17993
justinsb wants to merge 11 commits intokubernetes:masterfrom
justinsb:aiconform_integration

Conversation

@justinsb
Copy link
Member

@justinsb justinsb commented Feb 20, 2026

  • tests/ai-conformance: fix syntax for enabling gatewayAPI with cilium
  • tests/ai-conformance: update path to nvidia device plugin
  • tests/ai-conformance: don't panic if no zones for AWS
  • tests/ai-conformance: Update DRA driver installation
  • tests/ai-conformance: use server-side-apply for ray installation

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 20, 2026
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Feb 20, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign olemarkus for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from zetaab February 20, 2026 20:08
@justinsb justinsb changed the title aiconform integration WIP: AI Conformance testing Feb 20, 2026
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 20, 2026
@justinsb
Copy link
Member Author

justinsb commented Feb 20, 2026

/test pull-kops-ai-conformance

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/provider/gcp Issues or PRs related to gcp provider and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 20, 2026
ameukam
ameukam reviewed Feb 20, 2026
View reviewed changes
tests/e2e/kubetest2-kops/deployer/common.go Outdated
if d.region == "" {
// Default to us-east1, but use the region implied by the first zone if possible
if len(d.zones) == 0 {
d.region = "us-east1"
Copy link
Member

@ameukam ameukam Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer us-east2 as a region less subject to interruptions.

Copy link
Member Author

@justinsb justinsb Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool - split it out as #17994

@justinsb
Copy link
Member Author

justinsb commented Feb 20, 2026

/test pull-kops-ai-conformance

@justinsb justinsb force-pushed the aiconform_integration branch from b380b57 to a6e7a80 Compare February 20, 2026 22:46
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 20, 2026
@justinsb
Copy link
Member Author

justinsb commented Feb 20, 2026

/test pull-kops-ai-conformance

@justinsb justinsb force-pushed the aiconform_integration branch from 5eff7af to c5c2770 Compare February 20, 2026 23:43
@justinsb
Copy link
Member Author

justinsb commented Feb 20, 2026

/test pull-kops-ai-conformance

9 similar comments
@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@ameukam
Copy link
Member

ameukam commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@justinsb
Copy link
Member Author

justinsb commented Feb 21, 2026

/test pull-kops-ai-conformance

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 21, 2026
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2026
@justinsb justinsb force-pushed the aiconform_integration branch from 3615d30 to 07e9b55 Compare February 22, 2026 14:54
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2026
@justinsb
Copy link
Member Author

justinsb commented Feb 22, 2026

/test pull-kops-ai-conformance

ameukam
ameukam reviewed Feb 22, 2026
View reviewed changes
@justinsb
Copy link
Member Author

justinsb commented Feb 22, 2026

/test pull-kops-ai-conformance

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 22, 2026
@justinsb justinsb force-pushed the aiconform_integration branch from d174917 to 849e8c7 Compare February 23, 2026 02:04
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 23, 2026
@justinsb justinsb force-pushed the aiconform_integration branch 2 times, most recently from b0e9fd6 to cb336b1 Compare February 23, 2026 12:59
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 23, 2026
@justinsb
tests/ai-conformance: update ray installation
eb5c404 
The CRDs for Ray are too large to be installed with client-side apply,
we hit the annotation size limit; use server-side apply instead.

Also the default config installs to the "default" namespace,
so use the version with webhooks to install to ray-system instead.

That requires cert-manager though, so add that as well.
@justinsb
tests/ai-conformance: use GPU Operator
8a1c07d
@justinsb
tests/ai-conformance: validate Gateway CRD is installed
8325118
@justinsb
tests/ai-conformance: work with zones
030760e
@justinsb
tests/ai-conformance: create with normal nodes, then add GPU nodes
792e99d
@justinsb
tests/ai-conformance: export CLUSTER_NAME from kubetest2
fa72c08
@justinsb
tests/ai-conformance: set up helm early
aaf9853
@justinsb
tests/ai-conformance: simple test and harness for DRA compliance
e9ae379
@justinsb
tests/ai-conformance: swap order of ray and kueue
b0acec4
@justinsb
tests/ai-conformance: add toleration for nvidia.com/gpu taint
6451ae3
@justinsb
tests/ai-conformance: use nvidia GPU operator to install GPU drivers
8e1afe8
@justinsb justinsb force-pushed the aiconform_integration branch from cb336b1 to 8e1afe8 Compare February 24, 2026 16:28
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 24, 2026
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Feb 24, 2026

@justinsb: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-verify-boilerplate 8e1afe8 link true /test pull-kops-verify-boilerplate

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Mar 7, 2026

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@johngmyers johngmyers Awaiting requested review from johngmyers

@zetaab zetaab Awaiting requested review from zetaab

2 more reviewers

@ameukam ameukam ameukam left review comments

@upodroid upodroid upodroid left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area/provider/gcp Issues or PRs related to gcp provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@justinsb @k8s-ci-robot @ameukam @upodroid