Skip to content

Add cluster health tests#551

Merged
dbasunag merged 1 commit intoopendatahub-io:mainfrom
dbasunag:cluster_sanity
Aug 25, 2025
Merged

Add cluster health tests#551
dbasunag merged 1 commit intoopendatahub-io:mainfrom
dbasunag:cluster_sanity

Conversation

@dbasunag
Copy link
Copy Markdown
Collaborator

@dbasunag dbasunag commented Aug 25, 2025

Description

How Has This Been Tested?

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

Summary by CodeRabbit

  • Tests
    • Added a new cluster health test suite covering node readiness, schedulability, and data science cluster initialization and readiness.
  • Chores
    • Introduced a new test marker “cluster_health” to enable selective test runs (use -m cluster_health).

@dbasunag dbasunag requested a review from a team as a code owner August 25, 2025 13:08
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Aug 25, 2025

📝 Walkthrough

Walkthrough

Adds a new pytest marker entry (cluster_health) in pytest.ini and introduces a new test module with three cluster health tests covering node readiness/schedulability, Data Science Cluster Initialization readiness, and DataScienceCluster readiness using existing fixtures and utility functions.

Changes

Cohort / File(s) Summary of changes
Pytest configuration
pytest.ini
Added new marker cluster_health to the markers list; no other configuration changes.
Cluster health tests
tests/cluster_health/test_cluster_health.py
New test module with three tests: node health and schedulability checks using assert_nodes_in_healthy_condition and assert_nodes_schedulable; DS initialization readiness via wait_for_dsci_status_ready; DataScienceCluster readiness via wait_for_dsc_status_ready. Uses fixtures nodes, dsci_resource, dsc_resource.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions
Copy link
Copy Markdown

The following are automatically added/executed:

  • PR size label.
  • Run pre-commit
  • Run tox
  • Add PR author as the PR assignee
  • Build image based on the PR

Available user actions:

  • To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
  • To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
  • To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
    lgtm label removed on each new commit push.
  • To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
    verified label removed on each new commit push.
  • To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
    and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
  • To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
    pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.
Supported labels

{'/lgtm', '/hold', '/cherry-pick', '/build-push-pr-image', '/wip', '/verified'}

Copy link
Copy Markdown
Contributor

@fege fege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
pytest.ini (1)

21-21: Tighten grammar and align with existing marker phrasing

Other markers use “Mark tests …”. Let’s keep it consistent and fix the verb agreement.

-    cluster_health: Tests that verifies that cluster is healthy to begin testing
+    cluster_health: Mark tests that verify the cluster is healthy before running the suite
tests/cluster_health/test_cluster_health.py (2)

16-23: Make expectations explicit and consider adding a short polling window to reduce flakes

The helpers return True or raise; asserting the True return clarifies intent. Also, per utilities/infra.py, these “wait_for_*” helpers don’t actually poll; they check once and raise. That can be flaky immediately after cluster bring-up.

Minimal explicitness:

-    wait_for_dsci_status_ready(dsci_resource=dsci_resource)
+    assert wait_for_dsci_status_ready(dsci_resource=dsci_resource)
@@
-    wait_for_dsc_status_ready(dsc_resource=dsc_resource)
+    assert wait_for_dsc_status_ready(dsc_resource=dsc_resource)

Optional flake reduction (illustrative only; adapt if you prefer a shared retry utility):

# Example pattern if you decide to poll here:
import time
from utilities.exceptions import ResourceNotReadyError

def _assert_ready_with_retry(check_fn, *, timeout_s=300, interval_s=10):
    end = time.time() + timeout_s
    last_err = None
    while time.time() < end:
        try:
            assert check_fn()
            return
        except ResourceNotReadyError as e:
            last_err = e
            time.sleep(interval_s)
    raise last_err if last_err else AssertionError("Resource not ready before timeout")

11-11: Type hint portability across Python versions

The annotation list[Node] requires Python 3.9+. If your supported matrix includes Python 3.8, switch to typing.List.

+from typing import List
@@
-def test_cluster_node_healthy(nodes: list[Node]):
+def test_cluster_node_healthy(nodes: List[Node]):
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between d7e08d9 and 5770bf2.

📒 Files selected for processing (2)
  • pytest.ini (1 hunks)
  • tests/cluster_health/test_cluster_health.py (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/cluster_health/test_cluster_health.py (2)
utilities/infra.py (2)
  • wait_for_dsci_status_ready (1042-1049)
  • wait_for_dsc_status_ready (1057-1063)
tests/conftest.py (3)
  • nodes (560-561)
  • dsci_resource (410-411)
  • dsc_resource (415-416)

Comment thread tests/cluster_health/test_cluster_health.py
Comment thread tests/cluster_health/test_cluster_health.py
@dbasunag dbasunag merged commit 2195921 into opendatahub-io:main Aug 25, 2025
11 checks passed
@dbasunag dbasunag deleted the cluster_sanity branch August 25, 2025 13:55
@dbasunag
Copy link
Copy Markdown
Collaborator Author

/cherry-pick 2.24

rhods-ci-bot pushed a commit that referenced this pull request Aug 25, 2025
@rhods-ci-bot
Copy link
Copy Markdown
Contributor

Cherry pick action created PR #552 successfully 🎉!
See: https://github.com/opendatahub-io/opendatahub-tests/actions/runs/17210976498

@dbasunag
Copy link
Copy Markdown
Collaborator Author

/cherry-pick 2.23

@rhods-ci-bot
Copy link
Copy Markdown
Contributor

Cherry pick action created PR #553 successfully 🎉!
See: https://github.com/opendatahub-io/opendatahub-tests/actions/runs/17210990314

@github-actions
Copy link
Copy Markdown

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

dbasunag added a commit that referenced this pull request Aug 25, 2025
Co-authored-by: Debarati Basu-Nag <dbasunag@redhat.com>
@dbasunag
Copy link
Copy Markdown
Collaborator Author

/cherry-pick 2.22

rhods-ci-bot pushed a commit that referenced this pull request Aug 25, 2025
@rhods-ci-bot
Copy link
Copy Markdown
Contributor

Cherry pick action created PR #555 successfully 🎉!
See: https://github.com/opendatahub-io/opendatahub-tests/actions/runs/17212573238

dbasunag added a commit that referenced this pull request Aug 25, 2025
Co-authored-by: Debarati Basu-Nag <dbasunag@redhat.com>
dbasunag added a commit that referenced this pull request Aug 25, 2025
Co-authored-by: Debarati Basu-Nag <dbasunag@redhat.com>
@dbasunag
Copy link
Copy Markdown
Collaborator Author

/cherry-pick 2.21

@rhods-ci-bot
Copy link
Copy Markdown
Contributor

Cherry pick action created PR #566 successfully 🎉!
See: https://github.com/opendatahub-io/opendatahub-tests/actions/runs/17300198337

dbasunag added a commit that referenced this pull request Aug 28, 2025
* Add cluster health tests (#551)

* update docker file

---------

Co-authored-by: Debarati Basu-Nag <dbasunag@redhat.com>
mwaykole pushed a commit to mwaykole/opendatahub-tests that referenced this pull request Jan 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants