feat(evalhub): add EvalHub service health check tests by ruivieira · Pull Request #1145 · opendatahub-io/opendatahub-tests

ruivieira · 2026-02-25T08:50:52Z

Summary

Add initial test coverage for the EvalHub service, including:

Health endpoint smoke test via kube-rbac-proxy
EvalHub CR, Deployment, Route, and CA bundle fixtures
EvalHub resource wrapper in utilities/resources/

Related Issues

JIRA: https://issues.redhat.com/browse/RHOAIENG-50762

How it has been tested

Locally
Jenkins

Additional Requirements

If this PR introduces a new test image, did you create a PR to mirror it in disconnected environment?
If this PR introduces new marker(s)/adds a new component, was relevant ticket created to update relevant Jenkins job?

Summary by CodeRabbit

Tests
- Added a health check test for the EvalHub service to verify endpoint health and readiness.
Chores
- Added pytest fixtures to manage EvalHub lifecycle in tests (CR, deployment, route, TLS bundle).
- Added configuration constants for EvalHub service and CRD metadata.
- Added utilities to validate EvalHub health over HTTPS.
- Added a resource wrapper for EvalHub custom resources.

github-actions · 2026-02-25T08:51:12Z

The following are automatically added/executed:

PR size label.
Run pre-commit
Run tox
Add PR author as the PR assignee
Build image based on the PR

Available user actions:

To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
lgtm label removed on each new commit push.
To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
verified label removed on each new commit push.
To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.

Supported labels

{'/lgtm', '/wip', '/verified', '/hold', '/cherry-pick', '/build-push-pr-image'}

coderabbitai · 2026-02-25T08:56:04Z

📝 Walkthrough

Walkthrough

Adds test infrastructure and utilities for EvalHub: new pytest fixtures to manage EvalHub CR lifecycle, constants and a health-check utility, a health test, and a NamespacedResource wrapper class for the EvalHub custom resource.

Changes

Cohort / File(s)	Summary
EvalHub Test Fixtures & Constants `tests/model_explainability/evalhub/conftest.py`, `tests/model_explainability/evalhub/constants.py`	Adds pytest fixtures `evalhub_cr`, `evalhub_deployment`, `evalhub_route`, `evalhub_ca_bundle_file` to create/wait for CR, Deployment, Route, and CA bundle; introduces EvalHub-related constants (service, ports, paths, CRD metadata).
EvalHub Health Tests & Utils `tests/model_explainability/evalhub/test_evalhub_health.py`, `tests/model_explainability/evalhub/utils.py`	Adds `TestEvalHubHealth.test_evalhub_health_endpoint` and `validate_evalhub_health` function which performs an HTTPS GET to the EvalHub health endpoint with auth and CA bundle verification and asserts expected JSON fields/status.
EvalHub Resource Wrapper `utilities/resources/evalhub.py`	Adds `EvalHub` class extending `NamespacedResource` with API group set to TRUSTYAI_OPENDATAHUB_IO, optional `env` and `replicas` in constructor, and `to_dict` override injecting spec fields when provided.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding EvalHub service health check tests, which aligns with the core additions of test fixtures, health check test, and resource wrapper.
Description check	✅ Passed	The description follows the repository template with all required sections: Summary, Related Issues, How it has been tested, and Additional Requirements. It provides clear, actionable information about the changes.
Docstring Coverage	✅ Passed	Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings (stacked PR)
📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🧹 Nitpick comments (1)

utilities/resources/evalhub.py (1)
3-6: Avoid utility-layer imports from tests.* modules.

At Line 3, utilities/resources/evalhub.py depends on tests.model_explainability.evalhub.constants, which creates a reversed module boundary. Please move shared EvalHub CR metadata to a non-test shared module and import from there in both places.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@utilities/resources/evalhub.py` around lines 3 - 6, The file
utilities/resources/evalhub.py must not import from
tests.model_explainability.evalhub.constants; extract the shared CR metadata
constants (EVALHUB_KIND, EVALHUB_PLURAL) into a new non-test shared module
(e.g., a module named evalhub_constants or shared.evalhub_constants) and update
both utilities/resources/evalhub.py and
tests.model_explainability.evalhub.constants to import EVALHUB_KIND and
EVALHUB_PLURAL from that new shared module instead of from tests.* so the
utility layer no longer depends on test code.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/model_explainability/evalhub/conftest.py`:
- Around line 64-69: The fixture evalhub_ca_bundle_file may return an empty path
from create_ca_bundle_file(admin_client, ca_type="openshift"), causing later TLS
checks to fail with opaque errors; update the evalhub_ca_bundle_file fixture to
immediately fail-fast by asserting the returned CA bundle path is non-empty and
the file exists (e.g., check truthiness and os.path.exists/Path.exists on the
returned value) so failures point to the real missing OpenShift CA bundle;
reference create_ca_bundle_file and admin_client when adding these assertions.

---

Nitpick comments:
In `@utilities/resources/evalhub.py`:
- Around line 3-6: The file utilities/resources/evalhub.py must not import from
tests.model_explainability.evalhub.constants; extract the shared CR metadata
constants (EVALHUB_KIND, EVALHUB_PLURAL) into a new non-test shared module
(e.g., a module named evalhub_constants or shared.evalhub_constants) and update
both utilities/resources/evalhub.py and
tests.model_explainability.evalhub.constants to import EVALHUB_KIND and
EVALHUB_PLURAL from that new shared module instead of from tests.* so the
utility layer no longer depends on test code.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 2b3a900 and 103d085.

📒 Files selected for processing (6)

tests/model_explainability/evalhub/__init__.py
tests/model_explainability/evalhub/conftest.py
tests/model_explainability/evalhub/constants.py
tests/model_explainability/evalhub/test_evalhub_health.py
tests/model_explainability/evalhub/utils.py
utilities/resources/evalhub.py

dbasunag

Resource can be generated in a follow up PR too. Just let me know what you need from my side.

kpunwatk · 2026-02-25T17:14:58Z

Thanks for the PR, @ruivieira! This looks great—I've just left one small comment.
FYI, Before we merge any PR, we ensure run the tests locally and added the /verified label so we know it’s safe to go.

ruivieira · 2026-02-25T17:57:56Z

Thanks @dbasunag @kpunwatk the EvalHub resource is now generated from the CRD too.

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@utilities/resources/evalhub.py`:
- Around line 38-40: The code currently unconditionally overwrites
self.res["spec"] with an empty dict when neither self.kind_dict nor
self.yaml_file are present, which can drop existing spec fields; change this to
non-destructive initialization (e.g., use self.res.setdefault("spec", {}) or
guard with if "spec" not in self.res: self.res["spec"] = {}) so existing spec
content is preserved, then assign _spec = self.res["spec"] as before in the
block that references _spec.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 103d085 and 9bdae4c.

📒 Files selected for processing (1)

utilities/resources/evalhub.py

sheltoncyril

/lgtm

sheltoncyril

/lgtm

github-actions · 2026-02-26T10:49:55Z

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

feat(evalhub): add EvalHub service health check tests

103d085

ruivieira requested review from a team, kpunwatk and sheltoncyril as code owners February 25, 2026 08:50

github-actions Bot added ModelExplainability Utilities labels Feb 25, 2026

github-actions Bot assigned ruivieira Feb 25, 2026

github-actions Bot added the size/l label Feb 25, 2026

coderabbitai Bot reviewed Feb 25, 2026

View reviewed changes

Comment thread tests/model_explainability/evalhub/conftest.py

rhods-ci-bot added the commented-by-ruivieira label Feb 25, 2026

dbasunag previously approved these changes Feb 25, 2026

View reviewed changes

Comment thread utilities/resources/evalhub.py Outdated

rhods-ci-bot added the lgtm-by-dbasunag label Feb 25, 2026

kpunwatk reviewed Feb 25, 2026

View reviewed changes

Comment thread utilities/resources/evalhub.py

Merge branch 'main' into evalhub

f9aaaca

rhods-ci-bot added commented-by-kpunwatk and removed lgtm-by-dbasunag commented-by-ruivieira labels Feb 25, 2026

rhods-ci-bot added the Verified Verified pr in Jenkins label Feb 25, 2026

ruivieira added 2 commits February 25, 2026 17:54

fix: EvalHub resource generated from CRD

8130eeb

Merge remote-tracking branch 'origin/evalhub' into evalhub

9bdae4c

ruivieira dismissed dbasunag’s stale review via 9bdae4c February 25, 2026 17:57

rhods-ci-bot removed Verified Verified pr in Jenkins commented-by-kpunwatk labels Feb 25, 2026

rhods-ci-bot added the commented-by-ruivieira label Feb 25, 2026

coderabbitai Bot reviewed Feb 25, 2026

View reviewed changes

Comment thread utilities/resources/evalhub.py

dbasunag approved these changes Feb 25, 2026

View reviewed changes

Merge branch 'main' into evalhub

d99681d

rhods-ci-bot added lgtm-by-dbasunag commented-by-dbasunag and removed commented-by-ruivieira labels Feb 25, 2026

Merge branch 'main' into evalhub

38b85c7

rhods-ci-bot removed commented-by-dbasunag lgtm-by-dbasunag labels Feb 26, 2026

sheltoncyril approved these changes Feb 26, 2026

View reviewed changes

rhods-ci-bot added the lgtm-by-sheltoncyril label Feb 26, 2026

sheltoncyril approved these changes Feb 26, 2026

View reviewed changes

Comment thread utilities/resources/evalhub.py

sheltoncyril merged commit 8a28c9b into opendatahub-io:main Feb 26, 2026
11 checks passed

Conversation

ruivieira commented Feb 25, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issues

How it has been tested

Additional Requirements

Summary by CodeRabbit

Uh oh!

github-actions Bot commented Feb 25, 2026

Uh oh!

coderabbitai Bot commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dbasunag left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kpunwatk commented Feb 25, 2026

Uh oh!

ruivieira commented Feb 25, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sheltoncyril left a comment

Choose a reason for hiding this comment

Uh oh!

sheltoncyril left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ruivieira commented Feb 25, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Feb 25, 2026 •

edited

Loading

dbasunag left a comment •

edited

Loading