Refactor and Optimize E2E Tests

[ugiordan]

RHOAIENG-20490

Description

The current E2E tests can be optimized to reduce redundancy, improve consistency, and enhance debugging. This task involves:

• Using Gomega’s matchers properly for better readability.
• Extracting common logic into reusable helper functions.
• Ensuring stable and efficient resource handling with Eventually.
• Standardizing constants for namespaces and operators.

There are two new flags to configure the applications namepsace and delete the DSCI/DSC on test failures:

--delete-on-failure=true|false
--applications-namespace=opendatahub

How Has This Been Tested?

When running the tests with the following command:

make e2e-test E2E_TEST_FLAGS='-timeout 25m --applications-namespace=opendatahub --test-operator-controller=false --test-webhook=false --test-component=trustyai --test-service=!auth --delete-on-failure=true'

if an issue occurs during the test, the output is as follows:

go test ./tests/e2e/ -run ^TestOdhOperator -v --operator-namespace=opendatahub-operator-system -timeout 25m --applications-namespace=opendatahub --test-operator-controller=false --test-webhook=false --test-component=trustyai --test-service=!auth --delete-on-failure=true
=== RUN   TestOdhOperator
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-operators/servicemeshoperator_Operator_is_installed
=== PAUSE TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-operators/servicemeshoperator_Operator_is_installed
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-serverless/serverless-operator_Operator_is_installed
=== PAUSE TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-serverless/serverless-operator_Operator_is_installed
=== CONT  TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-operators/servicemeshoperator_Operator_is_installed
=== CONT  TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-serverless/serverless-operator_Operator_is_installed
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_creation_of_DSCInitialization_instance
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_creation_of_DataScienceCluster_instance
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_ServiceMeshSpec_in_DSCInitialization_instance
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_Knative_resource
=== RUN   TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_owned_namespaces_exist
=== RUN   TestOdhOperator/components
    controller_test.go:313: Skipping tests for components/codeflare
    controller_test.go:313: Skipping tests for components/kserve
    controller_test.go:313: Skipping tests for components/modelmeshserving
    controller_test.go:313: Skipping tests for components/dashboard
    controller_test.go:313: Skipping tests for components/datasciencepipelines
=== RUN   TestOdhOperator/components/trustyai
=== RUN   TestOdhOperator/components/trustyai/Enable_Kserve
=== NAME  TestOdhOperator/components/trustyai
    helper_test.go:288:
        Timed out after 120.001s.
        The function passed to Eventually failed at /Users/ugogiordano/workdir/rhoai/opendatahub-io/opendatahub-operator/tests/e2e/helper_test.go:1560 with:
        Expected resource 'default-kserve' of kind 'Kserve' to match condition {.status.conditions[] | select(.type == "Ready") | .status == "True" []}.
        cannot iterate over: null
=== NAME  TestOdhOperator/components/trustyai/Enable_Kserve
    testing.go:1577: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
=== NAME  TestOdhOperator/components
    controller_test.go:313: Skipping tests for components/kueue
    controller_test.go:313: Skipping tests for components/trainingoperator
    controller_test.go:313: Skipping tests for components/workbenches
    controller_test.go:313: Skipping tests for components/modelcontroller
    controller_test.go:313: Skipping tests for components/feastoperator
    controller_test.go:313: Skipping tests for components/ray
    controller_test.go:313: Skipping tests for components/modelregistry
=== RUN   TestOdhOperator/services
=== RUN   TestOdhOperator/services/monitoring
    controller_test.go:317: Monitoring ManagementState is 'Removed', skipping all monitoring-related tests.
=== NAME  TestOdhOperator/services
    controller_test.go:313: Skipping tests for services/auth
=== RUN   TestOdhOperator/trustyAITestSuite
=== RUN   TestOdhOperator/trustyAITestSuite/Enable_Kserve
=== NAME  TestOdhOperator/trustyAITestSuite
    helper_test.go:288:
        Timed out after 120.001s.
        The function passed to Eventually failed at /Users/ugogiordano/workdir/rhoai/opendatahub-io/opendatahub-operator/tests/e2e/helper_test.go:1560 with:
        Expected resource 'default-kserve' of kind 'Kserve' to match condition {.status.conditions[] | select(.type == "Ready") | .status == "True" []}.
        cannot iterate over: null
=== NAME  TestOdhOperator/trustyAITestSuite/Enable_Kserve
    testing.go:1577: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
=== NAME  TestOdhOperator
    controller_test.go:223: Cleaning up DataScienceCluster
    controller_test.go:223: Cleaning up DSCInitialization
--- FAIL: TestOdhOperator (280.53s)
    --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests (37.20s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed (0.00s)
            --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-operators/servicemeshoperator_Operator_is_installed (2.80s)
            --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Ensure_Service_Mesh_and_Serverless_operators_are_installed/Ensure_openshift-serverless/serverless-operator_Operator_is_installed (2.80s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_creation_of_DSCInitialization_instance (31.15s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_creation_of_DataScienceCluster_instance (2.75s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_ServiceMeshSpec_in_DSCInitialization_instance (0.16s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_Knative_resource (0.17s)
        --- PASS: TestOdhOperator/DSCInitialization_and_DataScienceCluster_management_E2E_Tests/Validate_owned_namespaces_exist (0.16s)
    --- FAIL: TestOdhOperator/components (120.59s)
        --- FAIL: TestOdhOperator/components/trustyai (120.59s)
            --- FAIL: TestOdhOperator/components/trustyai/Enable_Kserve (120.57s)
    --- PASS: TestOdhOperator/services (0.35s)
        --- SKIP: TestOdhOperator/services/monitoring (0.35s)
    --- FAIL: TestOdhOperator/trustyAITestSuite (121.25s)
        --- FAIL: TestOdhOperator/trustyAITestSuite/Enable_Kserve (121.21s)
FAIL
FAIL	github.com/opendatahub-io/opendatahub-operator/v2/tests/e2e	281.621s
FAIL
make: *** [e2e-test] Error 1

Screenshot or short clip

Merge criteria

• You have read the contributors guide.
• Commit messages are meaningful - have a clear and concise summary and detailed explanation of what was changed and why.
• Pull Request contains a description of the solution, a link to the JIRA issue, and to any dependent or related Pull Request.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[codecov]

Codecov Report

Attention: Patch coverage is 31.53846% with 89 lines in your changes missing coverage. Please review.

Project coverage is 24.01%. Comparing base (06a1b25) to head (a6050f5).

Files with missing lines	Patch %	Lines
pkg/utils/test/testf/testf_witht.go	21.05%	60 Missing ⚠️
pkg/resources/resources.go	56.81%	16 Missing and 3 partials ⚠️
pkg/utils/test/testf/testf_support.go	0.00%	10 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1738      +/-   ##
==========================================
+ Coverage   23.97%   24.01%   +0.03%     
==========================================
  Files         172      172              
  Lines       11699    11815     +116     
==========================================
+ Hits         2805     2837      +32     
- Misses       8614     8694      +80     
- Partials      280      284       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ajayjagan for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ugiordan]

/retest

[openshift-ci]

@ugiordan: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/opendatahub-operator-e2e	92bd128	link	true	/test opendatahub-operator-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.