Wait for pods to be in running state before attempting to create ModelRegistry

[dbasunag]

Without this change we see ModelRegistry post call gets x509: certificate has expired or is not yet valid once in a while.

Description

How Has This Been Tested?

Merge criteria:

• The commits are squashed in a cohesive manner and have meaningful messages.
• Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• The developer has manually tested the changes and verified that the changes work

Summary by Sourcery

Add a wait condition in test setup to ensure pods are fully running before proceeding with Model Registry tests.

Bug Fixes:

• Fix intermittent x509 certificate errors by waiting for application pods to be ready before tests interact with the Model Registry.

Enhancements:

• Add utility functions to check pod container statuses and determine if pods are in a running state.
• Implement a timeout sampler utility to wait for all pods in a namespace to reach a stable running state.

Tests:

• Modify the updated_dsc_component_state_scope_class fixture to wait for pods in the application namespace to be running before yielding the DSC resource.

Summary by CodeRabbit

• Tests
  • Improved test reliability by ensuring all required pods are running before proceeding with model registry tests.
  • Added utilities for checking pod status and waiting for pods to be ready during test execution.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request introduces a waiting mechanism in the updated_dsc_component_state_scope_class fixture to ensure all pods in the applications_namespace are in a running or succeeded state before proceeding. This is intended to mitigate occasional x509: certificate has expired or is not yet valid errors encountered during ModelRegistry creation.

No diagrams generated as the changes look simple and do not need a visual representation.

File-Level Changes

Change	Details	Files
Add utility functions to check and wait for pod running status.	Add function to get container error status within a pod. Add function to identify pods that are not in a running or succeeded state. Add function to wait for all pods in a namespace to reach a running or succeeded state using a timeout sampler.	tests/model_registry/utils.py
Integrate the new pod waiting mechanism into the updated_dsc_component_state_scope_class fixture.	Add admin_client argument to the fixture. Call the wait_for_pods_running function after ensuring the namespace is active. Specify the applications_namespace and 6 consecutive checks for the wait.	tests/model_registry/conftest.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[github-actions]

The following are automatically added/executed:

• PR size label.
• Run pre-commit
• Run tox
• Add PR author as the PR assignee

Available user actions:

• To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
• To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
• To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
  lgtm label removed on each new commit push.
• To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
  verified label removed on each new commit push.
• To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
  and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.

Supported labels

{'/hold', '/wip', '/verified', '/lgtm'}

[sourcery-ai]

Hey @dbasunag - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider moving the generic pod status checking functions (get_pod_container_error_status, get_not_running_pods, wait_for_pods_running) to a more general Kubernetes utility module.
• Consider defining the timeout, sleep interval, and consecutive check count used in wait_for_pods_running as constants instead of magic numbers.

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[coderabbitai]

Walkthrough

The changes introduce new utility functions in the model registry testing utilities for monitoring Kubernetes Pod statuses, including error inspection and waiting for all Pods to reach a running or succeeded state. The conftest.py fixture for updating the Data Science Cluster component state is modified to accept an admin client and now waits for all Pods in a specified namespace to be running before proceeding. Logging and error handling are enhanced for Pod status checks, and relevant imports and function signatures are updated accordingly.

Changes

File(s)	Change Summary
tests/model_registry/conftest.py	Updated imports to include wait_for_pods_running. Modified the updated_dsc_component_state_scope_class fixture to accept an admin_client parameter and to wait for all Pods in the target namespace to be running before proceeding.
tests/model_registry/utils.py	Added utility functions: get_pod_container_error_status, get_not_running_pods, and wait_for_pods_running. Introduced logging, timeout handling, and related imports for Pod status monitoring.

Sequence Diagram(s)

sequenceDiagram
    participant Test as Test Fixture
    participant AdminClient as admin_client
    participant Utils as utils.py
    participant K8s as Kubernetes API

    Test->>AdminClient: Pass admin_client to fixture
    Test->>Utils: Call wait_for_pods_running(admin_client, namespace)
    Utils->>K8s: List Pods in namespace
    Utils->>K8s: Check Pod statuses
    loop Until all Pods running/succeeded for N checks
        Utils->>K8s: Re-check Pods
    end
    Utils-->>Test: Proceed if all Pods running

Loading

Poem

In the warren of tests, new helpers appear,
To watch over Pods and keep statuses clear.
With patient hops, we now wait and see,
That all are running, as healthy as can be.
Logs and warnings, a bunny's gentle guide—
For clusters and carrots, we check side by side! 🐇✨

---

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 35186a6 and cdb24e4.

📒 Files selected for processing (2)

• tests/model_registry/conftest.py (3 hunks)
• tests/model_registry/utils.py (2 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/model_registry/utils.py
• tests/model_registry/conftest.py

✨ Finishing Touches

• 📝 Generate Docstrings

---

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR.
• @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 3

♻️ Duplicate comments (1)

tests/model_registry/utils.py (1)

166-169: Convert the inline comment to a proper docstring (previous feedback still applies).

The descriptive comment belongs in a docstring so that IDEs and help() can surface it.
This request has been raised in earlier rounds but is still unresolved.

🧹 Nitpick comments (1)

tests/model_registry/conftest.py (1)

236-240: Consider asserting the result of wait_for_pods_running or handling timeout.

wait_for_pods_running returns True on success and raises on timeout, but its return value is ignored.
Capturing the boolean (or at least logging it) would make failures more explicit and future refactors safer.

-        wait_for_pods_running(
+        assert wait_for_pods_running(
             admin_client=admin_client,
             namespace_name=py_config["applications_namespace"],
             number_of_consecutive_checks=6,
-        )
+        ), "Pods failed to reach running state within the expected time"

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between daf13fb and 35186a6.

📒 Files selected for processing (2)

• tests/model_registry/conftest.py (3 hunks)
• tests/model_registry/utils.py (2 hunks)

🧰 Additional context used

🧬 Code Graph Analysis (2)

tests/model_registry/conftest.py (2)

tests/model_registry/utils.py (1)

• wait_for_pods_running (198-231)

tests/conftest.py (2)

• dsc_resource (339-340)
• admin_client (50-51)

tests/model_registry/utils.py (3)

utilities/exceptions.py (2)

• ProtocolNotSupportedError (8-13)
• TooManyServicesError (16-21)

utilities/constants.py (2)

• Protocols (87-94)
• Annotations (124-139)

tests/conftest.py (1)

• admin_client (50-51)

[fege]

/lgtm

[dbasunag]

/verified

[github-actions]

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.