Skip to content

QG refator smoke (reduce time)#1201

Merged
mwaykole merged 2 commits intoopendatahub-io:mainfrom
mwaykole:qg-refactor
Mar 11, 2026
Merged

QG refator smoke (reduce time)#1201
mwaykole merged 2 commits intoopendatahub-io:mainfrom
mwaykole:qg-refactor

Conversation

@mwaykole
Copy link
Copy Markdown
Member

@mwaykole mwaykole commented Mar 11, 2026

Pull Request

Summary

Related Issues

  • Fixes:
  • JIRA:

How it has been tested

  • Locally
  • Jenkins

Additional Requirements

  • If this PR introduces a new test image, did you create a PR to mirror it in disconnected environment?
  • If this PR introduces new marker(s)/adds a new component, was relevant ticket created to update relevant Jenkins job?

Summary by CodeRabbit

  • Tests
    • Updated test categorization markers across multiple KServe model serving test files, reassigning tests from sanity/smoke to tier1/tier2 tiers to reorganize test execution hierarchy and selection criteria.

Signed-off-by: Milind waykole <mwaykole@redhat.com>
@github-actions
Copy link
Copy Markdown

The following are automatically added/executed:

  • PR size label.
  • Run pre-commit
  • Run tox
  • Add PR author as the PR assignee
  • Build image based on the PR

Available user actions:

  • To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
  • To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
  • To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
    lgtm label removed on each new commit push.
  • To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
    verified label removed on each new commit push.
  • To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
    and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
  • To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
    pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.
Supported labels

{'/build-push-pr-image', '/wip', '/hold', '/lgtm', '/verified', '/cherry-pick'}

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 11, 2026

📝 Walkthrough

Walkthrough

This pull request updates pytest test markers across 20 test files in the model serving test suite, changing test categorization labels from sanity/smoke to tier1/tier2. No test logic, assertions, or execution behavior is modified—only test metadata for selection and categorization purposes.

Changes

Cohort / File(s) Summary
sanity → tier1 marker updates
tests/model_serving/model_server/kserve/inference_graph/test_inference_graph_raw.py, tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_env_vars_updates.py, tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_replicas_update.py, tests/model_serving/model_server/kserve/metrics/test_non_admin_users.py, tests/model_serving/model_server/kserve/kueue/test_kueue_isvc_raw.py
Module-level or class-level pytest marker changed from @pytest.mark.sanity to @pytest.mark.tier1; test selection scope modified without affecting test logic.
smoke → tier1 marker updates
tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py, tests/model_serving/model_server/kserve/components/test_model_serving_components.py, tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_pull_secret_updates.py, tests/model_serving/model_server/kserve/metrics/test_model_metrics.py, tests/model_serving/model_server/kserve/model_car/test_oci_image.py, tests/model_serving/model_server/kserve/raw_deployment/test_kserve_raw_routes_reconciliation.py, tests/model_serving/model_server/kserve/stop_resume/test_raw_stop_resume_model.py
Test markers changed from @pytest.mark.smoke to @pytest.mark.tier1 on class or method level; categorization updated without behavioral changes.
sanity → smoke marker updates
tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py
Test decorators changed from @pytest.mark.sanity to @pytest.mark.smoke at two locations; test categorization modified without functional logic changes.
sanity/smoke → tier1 multi-marker suite updates
tests/model_serving/model_server/kserve/components/kserve_dsc_deployment_mode/test_kserve_dsc_default_deployment_mode.py
Module-level pytestmark updated: [pytest.mark.sanity, pytest.mark.rawdeployment][pytest.mark.tier1, pytest.mark.rawdeployment]; rawdeployment marker preserved.
tier2 → tier3 marker updates
tests/model_serving/model_server/kserve/components/test_custom_resources.py
Module-level pytestmark changed from tier2 to tier3 while retaining slow and valid_aws_config fixtures; test tier classification adjusted.
tier1 → tier2 marker downgrades
tests/model_serving/model_server/kserve/negative/test_invalid_model_name.py, tests/model_serving/model_server/kserve/negative/test_malformed_json_payload.py, tests/model_serving/model_server/kserve/negative/test_missing_required_fields.py, tests/model_serving/model_server/kserve/negative/test_unsupported_content_type.py, tests/model_serving/model_server/kserve/negative/test_wrong_input_data_type.py
Class-level decorators downgraded from @pytest.mark.tier1 to @pytest.mark.tier2; test tier classification lowered for negative test cases.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Description check ⚠️ Warning PR description is entirely a blank template with no substantive content, linked issues, testing details, or explanation of marker migration rationale. Complete Summary section explaining pytest marker changes and their impact, provide Related Issues/JIRA links, confirm testing approach, and justify tier reassignments.
Title check ❓ Inconclusive Title is vague and generic, using 'refactor' and 'reduce time' without clearly describing the actual changes (pytest marker migrations). Replace with specific title describing the primary change: e.g., 'Migrate pytest markers from sanity/smoke to tier1/tier2 for test categorization' or similar.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_pull_secret_updates.py (1)

27-39: ⚠️ Potential issue | 🟡 Minor

Apply @pytest.mark.tier1 at class level or to all three test methods.

The marker is applied only to test_initial_pull_secret_set, but test_update_pull_secret and test_remove_pull_secret depend on fixtures that require test 1 to complete first. Running with -m tier1 will skip tests 2 and 3, breaking the fixture chain. Align with the codebase pattern used in similar test files (e.g., test_missing_required_fields.py, test_unsupported_content_type.py): apply tier markers at the class level via decorator stacking.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_pull_secret_updates.py`
around lines 27 - 39, The pytest marker `@pytest.mark.tier1` is only applied to
test_initial_pull_secret_set causing test_update_pull_secret and
test_remove_pull_secret to be skipped when running with -m tier1 and breaking
the fixture chain; fix by applying the marker at the class level (decorate the
test class that contains test_initial_pull_secret_set, test_update_pull_secret,
and test_remove_pull_secret) or alternatively add `@pytest.mark.tier1` to both
test_update_pull_secret and test_remove_pull_secret so all three tests
(test_initial_pull_secret_set, test_update_pull_secret, test_remove_pull_secret)
share the same tier1 marker as in similar test files.
🧹 Nitpick comments (1)
tests/model_serving/model_server/kserve/raw_deployment/test_kserve_raw_routes_reconciliation.py (1)

26-31: Marker change noted, but pre-existing typos in method name.

Method test_raw_onnx_rout_reconciliation and docstring contain "rout" instead of "route". Consider fixing in a follow-up.

Typo fix suggestion
-    def test_raw_onnx_rout_reconciliation(self, ovms_raw_inference_service):
+    def test_raw_onnx_route_reconciliation(self, ovms_raw_inference_service):
         """
         Verify that the KServe Raw ONNX model can be queried using REST
-        and ensure that the model rout reconciliation works correctly .
+        and ensure that the model route reconciliation works correctly.
         """
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/model_server/kserve/raw_deployment/test_kserve_raw_routes_reconciliation.py`
around lines 26 - 31, Rename the test and update its docstring to fix the typo
"rout" → "route": change the test function name
test_raw_onnx_rout_reconciliation to test_raw_onnx_route_reconciliation and
update the docstring sentence "model rout reconciliation" to "model route
reconciliation" (ensure any references to test_raw_onnx_rout_reconciliation in
fixtures/markers like ovms_raw_inference_service or tests referencing this name
are also updated).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py`:
- Line 13: Remove the class-level `@pytest.mark.smoke` decorator from the
TestKServeTokenAuthenticationRaw class so the smoke marker is only applied to
the individual happy-path test methods; keep the method-level `@pytest.mark.smoke`
on the happy-path tests (e.g., test_successful_raw_model_authentication) and
remove any redundant `@pytest.mark.smoke` from other methods such as
test_raw_disable_enable_authentication_no_pod_rollout so non-core tests
(test_disabled_raw_model_authentication,
test_re_enabled_raw_model_authentication) are no longer included in the smoke
gate.

---

Outside diff comments:
In
`@tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_pull_secret_updates.py`:
- Around line 27-39: The pytest marker `@pytest.mark.tier1` is only applied to
test_initial_pull_secret_set causing test_update_pull_secret and
test_remove_pull_secret to be skipped when running with -m tier1 and breaking
the fixture chain; fix by applying the marker at the class level (decorate the
test class that contains test_initial_pull_secret_set, test_update_pull_secret,
and test_remove_pull_secret) or alternatively add `@pytest.mark.tier1` to both
test_update_pull_secret and test_remove_pull_secret so all three tests
(test_initial_pull_secret_set, test_update_pull_secret, test_remove_pull_secret)
share the same tier1 marker as in similar test files.

---

Nitpick comments:
In
`@tests/model_serving/model_server/kserve/raw_deployment/test_kserve_raw_routes_reconciliation.py`:
- Around line 26-31: Rename the test and update its docstring to fix the typo
"rout" → "route": change the test function name
test_raw_onnx_rout_reconciliation to test_raw_onnx_route_reconciliation and
update the docstring sentence "model rout reconciliation" to "model route
reconciliation" (ensure any references to test_raw_onnx_rout_reconciliation in
fixtures/markers like ovms_raw_inference_service or tests referencing this name
are also updated).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 32b1abca-7350-48fb-ac88-ad8acb7b4e3e

📥 Commits

Reviewing files that changed from the base of the PR and between 68d9aef and 6948922.

📒 Files selected for processing (20)
  • tests/model_serving/model_server/kserve/authentication/test_kserve_token_authentication_raw.py
  • tests/model_serving/model_server/kserve/authentication/test_non_admin_users.py
  • tests/model_serving/model_server/kserve/components/kserve_dsc_deployment_mode/test_kserve_dsc_default_deployment_mode.py
  • tests/model_serving/model_server/kserve/components/test_custom_resources.py
  • tests/model_serving/model_server/kserve/components/test_model_serving_components.py
  • tests/model_serving/model_server/kserve/inference_graph/test_inference_graph_raw.py
  • tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_env_vars_updates.py
  • tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_pull_secret_updates.py
  • tests/model_serving/model_server/kserve/inference_service_configuration/test_isvc_replicas_update.py
  • tests/model_serving/model_server/kserve/kueue/test_kueue_isvc_raw.py
  • tests/model_serving/model_server/kserve/metrics/test_model_metrics.py
  • tests/model_serving/model_server/kserve/metrics/test_non_admin_users.py
  • tests/model_serving/model_server/kserve/model_car/test_oci_image.py
  • tests/model_serving/model_server/kserve/negative/test_invalid_model_name.py
  • tests/model_serving/model_server/kserve/negative/test_malformed_json_payload.py
  • tests/model_serving/model_server/kserve/negative/test_missing_required_fields.py
  • tests/model_serving/model_server/kserve/negative/test_unsupported_content_type.py
  • tests/model_serving/model_server/kserve/negative/test_wrong_input_data_type.py
  • tests/model_serving/model_server/kserve/raw_deployment/test_kserve_raw_routes_reconciliation.py
  • tests/model_serving/model_server/kserve/stop_resume/test_raw_stop_resume_model.py

@mwaykole mwaykole enabled auto-merge (squash) March 11, 2026 07:29
@mwaykole mwaykole disabled auto-merge March 11, 2026 10:01
@mwaykole mwaykole enabled auto-merge (rebase) March 11, 2026 10:01
@mwaykole mwaykole merged commit 7ffe046 into opendatahub-io:main Mar 11, 2026
9 checks passed
@mwaykole mwaykole deleted the qg-refactor branch March 11, 2026 10:34
@github-actions
Copy link
Copy Markdown

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants