Skip to content

Add test steps in test#1294

Merged
mwaykole merged 1 commit intoopendatahub-io:mainfrom
mwaykole:llmdf
Mar 25, 2026
Merged

Add test steps in test#1294
mwaykole merged 1 commit intoopendatahub-io:mainfrom
mwaykole:llmdf

Conversation

@mwaykole
Copy link
Copy Markdown
Member

@mwaykole mwaykole commented Mar 25, 2026

Pull Request

Summary

Related Issues

  • Fixes:
  • JIRA:

How it has been tested

  • Locally
  • Jenkins

Additional Requirements

  • If this PR introduces a new test image, did you create a PR to mirror it in disconnected environment?
  • If this PR introduces new marker(s)/adds a new component, was relevant ticket created to update relevant Jenkins job?

Summary by CodeRabbit

Release Notes

  • Tests
    • Improved test documentation with detailed docstrings across the KServe test suite, clarifying validation scenarios for model serving, authentication, lifecycle management, ingress routing, multi-node deployments, observability, and platform configurations.

Signed-off-by: Milind waykole <mwaykole@redhat.com>
@github-actions
Copy link
Copy Markdown

The following are automatically added/executed:

  • PR size label.
  • Run pre-commit
  • Run tox
  • Add PR author as the PR assignee
  • Build image based on the PR

Available user actions:

  • To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
  • To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
  • To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
    lgtm label removed on each new commit push.
  • To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
    verified label removed on each new commit push.
  • To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
    and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
  • To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
    pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.
Supported labels

{'/verified', '/cherry-pick', '/build-push-pr-image', '/lgtm', '/wip', '/hold'}

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 25, 2026

📝 Walkthrough

Walkthrough

This pull request adds descriptive docstrings to 24 test classes and several test methods across the KServe test suite. All changes are documentation-only; no functional logic, assertions, control flow, or test behavior was modified.

Changes

Cohort / File(s) Summary
Authentication & Access Control
tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py, tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py
Added class-level docstrings documenting end-to-end authentication flows, token validation scenarios, and non-admin user deployment steps.
Inference Validation
tests/model_serving/model_server/kserve/inference_graph/test_inference_graph_raw.py
Added class and method docstrings to document public/private raw deployment inference graph validation and auth-enabled test scenarios.
Lifecycle Management
tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_env_vars_updates.py, test_isvc_pull_secret_updates.py, test_isvc_replicas_update.py, test_raw_stop_resume_model.py
Added class and method docstrings describing environment variable updates, pull secret lifecycle, replica scaling, and stop/resume workflows. Consolidated multi-line assertion comments to single-line docstrings in stop/resume tests.
Ingress & Routing
tests/model_serving/model_server/kserve/ingress/test_route_reconciliation.py, test_route_visibility.py
Added class docstrings for route visibility tests across REST/gRPC protocols and timeout scenarios. Replaced inline comments with docstrings in reconciliation tests.
Multi-Node GPU Inference
tests/model_serving/model_server/kserve/multi_node/test_nvidia_multi_node.py, test_oci_multi_node.py
Added class docstrings documenting Ray-based vLLM multi-node workflows and OCI-backed model image deployments.
Observability & Metrics
tests/model_serving/model_server/kserve/observability/test_model_metrics.py, test_non_admin_users.py
Added class docstrings describing OVMS metrics validation via UserWorkloadMonitoring and Prometheus query flows for unprivileged users.
Platform & Components
tests/model_serving/model_server/kserve/platform/dsc_deployment_mode/test_kserve_dsc_default_deployment_mode.py, test_custom_resources.py, test_model_serving_components.py
Added class docstrings documenting DSC deployment mode propagation, custom resource state transitions, and KServe control-plane component deployment validation.
Storage Backends
tests/model_serving/model_server/kserve/storage/oci/test_oci_image.py, pvc/test_kserve_pvc_rwx.py, pvc/test_kserve_pvc_write_access.py, s3/test_s3_raw_deployment.py
Added class docstrings for OCI Model Car deployments, ReadWriteMany PVC multi-pod access validation, read-only annotation enforcement, and MinIO S3 inference workflows.
Upgrade Scenarios
tests/model_serving/model_server/upgrade/test_upgrade.py
Added class docstrings documenting pre-upgrade inference validation and post-upgrade integrity/inference checks.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is entirely a template with placeholder text and no actual implementation details, rationale, testing information, or concrete changes documented. Fill in the Summary section with a clear explanation of docstring additions across test files. Document what was tested and provide any relevant issue links.
Title check ❓ Inconclusive The title 'Add test steps in test' is vague and does not clearly summarize the actual changes, which are docstring additions to 20+ test files. Revise title to be more specific and descriptive, such as 'Add docstrings to test classes and methods' to clearly convey the nature of the changes.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py (1)

55-55: ⚠️ Potential issue | 🟡 Minor

Fix malformed assertion error message.

The assertion message is missing the f prefix, so it will print the literal string {pod.name for pod in isvc_pods} instead of evaluating the set comprehension when the assertion fails.

🐛 Proposed fix
-        assert len(isvc_pods) == 2, "Expected 2 inference pods, existing pods: {pod.name for pod in isvc_pods}"
+        assert len(isvc_pods) == 2, f"Expected 2 inference pods, existing pods: {[pod.name for pod in isvc_pods]}"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py`
at line 55, The assertion message is a normal string so the set comprehension is
not evaluated; update the assertion in the test (the line asserting
len(isvc_pods) == 2) to use an f-string (or format) so the expression {pod.name
for pod in isvc_pods} is interpolated into the message (e.g., add the leading f
to the string) to show the actual pod names when the assertion fails.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py`:
- Around line 42-50: The docstring and test name claim an increase operation but
the code only verifies initial 2 replicas then scales down to 1; update the
documentation and test name to match behavior: change the module/class docstring
to describe "deploy with 2 min-replicas, verify 2 pods, run inference, patch
ISVC to scale down to 1 and verify 1 pod and inference" and rename the test
function test_raw_increase_isvc_replicas to test_raw_decrease_isvc_replicas (or
similar) so the name reflects the actual assertion; locate references to
TestRawISVCReplicasUpdates and test_raw_increase_isvc_replicas in the file to
update them consistently.

---

Outside diff comments:
In
`@tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py`:
- Line 55: The assertion message is a normal string so the set comprehension is
not evaluated; update the assertion in the test (the line asserting
len(isvc_pods) == 2) to use an f-string (or format) so the expression {pod.name
for pod in isvc_pods} is interpolated into the message (e.g., add the leading f
to the string) to show the actual pod names when the assertion fails.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 8291715a-d1d0-40e3-9aec-38ec150727f9

📥 Commits

Reviewing files that changed from the base of the PR and between 850de4b and c251f03.

📒 Files selected for processing (21)
  • tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py
  • tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py
  • tests/model_serving/model_server/kserve/inference_graph/test_inference_graph_raw.py
  • tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_env_vars_updates.py
  • tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_pull_secret_updates.py
  • tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py
  • tests/model_serving/model_server/kserve/inference_service_lifecycle/test_raw_stop_resume_model.py
  • tests/model_serving/model_server/kserve/ingress/test_route_reconciliation.py
  • tests/model_serving/model_server/kserve/ingress/test_route_visibility.py
  • tests/model_serving/model_server/kserve/multi_node/test_nvidia_multi_node.py
  • tests/model_serving/model_server/kserve/multi_node/test_oci_multi_node.py
  • tests/model_serving/model_server/kserve/observability/test_model_metrics.py
  • tests/model_serving/model_server/kserve/observability/test_non_admin_users.py
  • tests/model_serving/model_server/kserve/platform/dsc_deployment_mode/test_kserve_dsc_default_deployment_mode.py
  • tests/model_serving/model_server/kserve/platform/test_custom_resources.py
  • tests/model_serving/model_server/kserve/platform/test_model_serving_components.py
  • tests/model_serving/model_server/kserve/storage/oci/test_oci_image.py
  • tests/model_serving/model_server/kserve/storage/pvc/test_kserve_pvc_rwx.py
  • tests/model_serving/model_server/kserve/storage/pvc/test_kserve_pvc_write_access.py
  • tests/model_serving/model_server/kserve/storage/s3/test_s3_raw_deployment.py
  • tests/model_serving/model_server/upgrade/test_upgrade.py

Comment on lines +42 to +50
"""Validate scaling replica count up and down on a KServe raw deployment ISVC.

Steps:
1. Deploy an OVMS inference service with 2 min-replicas and 4 max-replicas.
2. Verify that 2 predictor pods are running after deployment.
3. Run inference to confirm the model responds correctly with multiple replicas.
4. Patch the ISVC to scale down to 1 replica and verify only 1 pod remains.
5. Run inference again to confirm the model responds correctly after scale-down.
"""
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Docstring inaccurately describes the test flow.

The docstring implies scaling up (2→4) and down (4→1), but the tests only verify the initial deployment of 2 replicas and then scale down to 1. There's no actual scale-up operation tested. The class is named TestRawISVCReplicasUpdates and the first test is named test_raw_increase_isvc_replicas, but it only asserts the initial replica count rather than testing an increase operation.

📝 Proposed correction
-    """Validate scaling replica count up and down on a KServe raw deployment ISVC.
+    """Validate replica scaling behavior on a KServe raw deployment ISVC.
 
     Steps:
         1. Deploy an OVMS inference service with 2 min-replicas and 4 max-replicas.
         2. Verify that 2 predictor pods are running after deployment.
         3. Run inference to confirm the model responds correctly with multiple replicas.
         4. Patch the ISVC to scale down to 1 replica and verify only 1 pod remains.
         5. Run inference again to confirm the model responds correctly after scale-down.
     """
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/model_server/kserve/inference_service_lifecycle/test_isvc_replicas_update.py`
around lines 42 - 50, The docstring and test name claim an increase operation
but the code only verifies initial 2 replicas then scales down to 1; update the
documentation and test name to match behavior: change the module/class docstring
to describe "deploy with 2 min-replicas, verify 2 pods, run inference, patch
ISVC to scale down to 1 and verify 1 pod and inference" and rename the test
function test_raw_increase_isvc_replicas to test_raw_decrease_isvc_replicas (or
similar) so the name reflects the actual assertion; locate references to
TestRawISVCReplicasUpdates and test_raw_increase_isvc_replicas in the file to
update them consistently.

@mwaykole mwaykole merged commit 5196c21 into opendatahub-io:main Mar 25, 2026
9 checks passed
@github-actions
Copy link
Copy Markdown

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants