New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[sync] Improve e2e test configurability (#1650) #1732

Open

zdtsw wants to merge 4 commits into opendatahub-io:rhoai from zdtsw-forking:sync_pr_1650

Member

zdtsw commented Mar 12, 2025

Improve e2e test configurability

This feature is to improve the configurability of the e2e test suite, which is particularly useful when running the test suite locally.

There are two new flags to globally enabled/disable some test grouops such as components and services:

--test-components=true|false
--test-services=true|false

It is also possible to disable a particular test by preceding its name with an exlamation mark (!), i.e. to run only the test for the dashbaord component and skip the tests for the auth service:

make e2e-test E2E_TEST_FLAGS='--test-component=dashboard --test-service=!auth'

Fix validation logic

(cherry picked from commit 39da621)

Description

sync #1650

How Has This Been Tested?

Screenshot or short clip

Merge criteria

You have read the contributors guide.
Commit messages are meaningful - have a clear and concise summary and detailed explanation of what was changed and why.
Pull Request contains a description of the solution, a link to the JIRA issue, and to any dependent or related Pull Request.
Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
The developer has manually tested the changes and verified that the changes work

openshift-ci bot requested review from grdryn and ugiordan

March 12, 2025 07:45

zdtsw added testing rhoai labels

zdtsw requested review from lburgazzoli and removed request for grdryn and ugiordan

March 12, 2025 07:45

lburgazzoli approved these changes

View reviewed changes

openshift-ci bot assigned lburgazzoli

openshift-ci bot added lgtm approved labels

ugiordan approved these changes

View reviewed changes

openshift-ci bot assigned ugiordan

openshift-ci bot commented Mar 12, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lburgazzoli, ugiordan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [lburgazzoli,ugiordan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ugiordan reviewed

View reviewed changes

tests/e2e/cfmap_deletion_test.go

               }
               func createDeletionConfigMap(tc *testContext, enableDeletion string) error {
               	configMap := &corev1.ConfigMap{
               		ObjectMeta: metav1.ObjectMeta{
               			Name:      deleteConfigMap,
-              			Namespace: tc.operatorNamespace,
+              			Namespace: testOpts.operatorNamespace,

Member

ugiordan Mar 12, 2025

If we use testOpts directly inside a test without passing it through the context, it can cause an initialization cycle issue. This becomes problematic if you want to debug a specific component test without running the entire test suite.

For example, if I place this test inside TestOdhOperator for debugging it

t.Run("kueueTestSuite", kueueTestSuite)

I'll get the following issue:

# github.com/opendatahub-io/opendatahub-operator/v2/tests/e2e_test [github.com/opendatahub-io/opendatahub-operator/v2/tests/e2e.test]
./controller_test.go:42:2: initialization cycle for testOpts
	./controller_test.go:42:2: testOpts refers to
	./modelmeshserving_test.go:23:6: modelMeshServingTestSuite refers to
	./components_test.go:68:29: ValidateComponentEnabled refers to
	./controller_test.go:42:2: testOpts

Member

lburgazzoli commented Mar 12, 2025

/test opendatahub-operator-e2e

1 similar comment

Member

lburgazzoli commented Mar 12, 2025

/test opendatahub-operator-e2e

Member

lburgazzoli commented Mar 13, 2025

/retest

openshift-ci bot removed the lgtm label

openshift-ci bot commented Mar 13, 2025

New changes are detected. LGTM label has been removed.

Member Author

zdtsw commented Mar 14, 2025

/test opendatahub-operator-e2e

zdtsw force-pushed the sync_pr_1650 branch from ac34da7 to 54bd24a Compare

March 14, 2025 15:25

Member Author

zdtsw commented Mar 15, 2025

/test opendatahub-operator-e2e

zdtsw mentioned this pull request

fix: testcase for Auth #1751

Merged

5 tasks

Member Author

zdtsw commented Mar 17, 2025

/test opendatahub-operator-e2e

1 similar comment

Member Author

zdtsw commented Mar 18, 2025

/test opendatahub-operator-e2e

Member Author

zdtsw commented Mar 18, 2025

i cannot think of any reason after my last commits try to fix the test on RHOAI, it still failed with
expected odh-admins, found rhods-admins
the reason it falls into this case is
tc get release as ODH or Unknown and the value set in Auth is correct.

Member

lburgazzoli commented Mar 20, 2025

/test all

1 similar comment

Member

lburgazzoli commented Mar 21, 2025

/test all

openshift-merge-robot added the needs-rebase label

lburgazzoli and others added 4 commits

March 24, 2025 17:57


          Improve e2e test configurability (opendatahub-io#1650)

* Improve e2e test configurability

This feature is to improve the configurability of the e2e test suite,
which is particularly useful when running the test suite locally.

There are two new flags to globally enabled/disable some test grouops
such as components and services:

* --test-components=true|false
* --test-services=true|false

It is also possible to disable a particular test by preceding its name
with an exlamation mark (!), i.e. to run only the test for the dashbaord
component and skip the tests for the auth service:

  make e2e-test E2E_TEST_FLAGS='--test-component=dashboard --test-service=!auth'

* Fix validation logic

(cherry picked from commit 39da621)


          fix auth tests for RHOAI (opendatahub-io#1700)

142836a

* fix auth tests for RHOAI

* fix lint

---------

Co-authored-by: Steven Tobin <[email protected]>
(cherry picked from commit e8e430c)
Signed-off-by: Wen Zhou <[email protected]>


          fix: add missing changes in bundle + fix tests

ebdb7a2

- ODH it falls to Unknonw but in RHOAI it fails test with "" string

Signed-off-by: Wen Zhou <[email protected]>


          update: pass in platform type in CSV

711b5f7

- getPlatform can properly work
- otherwise, in RHOAI e2e, it return Unknown nil by detectManagedRhoai

Signed-off-by: Wen Zhou <[email protected]>

zdtsw force-pushed the sync_pr_1650 branch from 9de59d3 to 711b5f7 Compare

March 24, 2025 17:10

openshift-merge-robot removed the needs-rebase label

openshift-ci bot commented Mar 24, 2025

@zdtsw: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/opendatahub-operator-e2e	`711b5f7`	link	true	`/test opendatahub-operator-e2e`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

mlassak mentioned this pull request

[sync] update: remove Unknown constant from platform detection logic #1800

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved rhoai testing