Skip to content

tests(maas): add multiple auth policy and cascade deletion tests#1220

Merged
dbasunag merged 6 commits intoopendatahub-io:mainfrom
SB159:feature/test-multiple-auth-policies-per-model
Mar 17, 2026
Merged

tests(maas): add multiple auth policy and cascade deletion tests#1220
dbasunag merged 6 commits intoopendatahub-io:mainfrom
SB159:feature/test-multiple-auth-policies-per-model

Conversation

@SB159
Copy link
Copy Markdown
Contributor

@SB159 SB159 commented Mar 13, 2026

Pull Request

Summary

add multiple auth policy and cascade deletion tests

Related Issues

  • Fixes:
  • JIRA:

How it has been tested

  • Locally
  • Jenkins

Additional Requirements

  • If this PR introduces a new test image, did you create a PR to mirror it in disconnected environment?
  • If this PR introduces new marker(s)/adds a new component, was relevant ticket created to update relevant Jenkins job?

Summary by CodeRabbit

  • Tests
    • Added tests for subscription cascade deletion: ensures model access is rebuilt after removing a subscription and that deleting the last subscription denies access until restored.
    • Added tests for multiple auth policies per model: validates OR-style policy behavior—access granted if any matching policy/subscription exists and denied when removed.
    • Added test fixtures to create temporary subscriptions and auth-policy setups to support these scenarios.

@github-actions
Copy link
Copy Markdown

The following are automatically added/executed:

  • PR size label.
  • Run pre-commit
  • Run tox
  • Add PR author as the PR assignee
  • Build image based on the PR

Available user actions:

  • To mark a PR as WIP, add /wip in a comment. To remove it from the PR comment /wip cancel to the PR.
  • To block merging of a PR, add /hold in a comment. To un-block merging of PR comment /hold cancel.
  • To mark a PR as approved, add /lgtm in a comment. To remove, add /lgtm cancel.
    lgtm label removed on each new commit push.
  • To mark PR as verified comment /verified to the PR, to un-verify comment /verified cancel to the PR.
    verified label removed on each new commit push.
  • To Cherry-pick a merged PR /cherry-pick <target_branch_name> to the PR. If <target_branch_name> is valid,
    and the current PR is merged, a cherry-picked PR would be created and linked to the current PR.
  • To build and push image to quay, add /build-push-pr-image in a comment. This would create an image with tag
    pr-<pr_number> to quay repository. This image tag, however would be deleted on PR merge or close action.
Supported labels

{'/cherry-pick', '/build-push-pr-image', '/wip', '/lgtm', '/verified', '/hold'}

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 13, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds two new pytest modules and fixtures that test MaaS subscription and auth-policy behavior: cascade deletion/rebuild and OR-logic across multiple auth policies. Tests perform resource creation/deletion, readiness polling, and HTTP access assertions (expecting 200, 403, or 429).

Changes

Cohort / File(s) Summary
Cascade deletion tests
tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py
New test class TestCascadeDeletion with tests that create temporary MaaSSubscription resources, wait for Ready, delete/restore subscriptions, and poll model endpoints to assert access changes (expects HTTP 200 after rebuild, 403/429 when last subscription removed).
Multiple auth-policies tests
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py
New test class TestMultipleAuthPoliciesPerModel with three tests exercising OR-logic across MaaSAuthPolicy and MaaSSubscription resources: initial denial (403), allow when extra policy+subscription present (200), and denial after removing extra policy (403).
Test fixtures / imports
tests/model_serving/maas_billing/maas_subscription/conftest.py
Adds import create_maas_subscription and two fixtures: temporary_system_authenticated_subscription (creates temporary subscription owned by system:authenticated, waits Ready, cleans up) and premium_system_authenticated_access (creates an extra auth policy and matching subscription, waits Ready, yields objects, and cleans up auth policy).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The description uses the template structure but provides minimal substantive detail in the Summary section, leaving Related Issues, testing status, and additional requirements blank or unchecked. Add specific details to the Summary explaining what cascade deletion and multiple auth policy tests verify. Specify any related tickets or JIRA issues, and confirm testing status (Locally/Jenkins).
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately reflects the main changes: adding tests for multiple auth policy and cascade deletion behavior.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can use Trivy to scan for security misconfigurations and secrets in Infrastructure as Code files.

Add a .trivyignore file to your project to customize which findings Trivy reports.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (5)
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py (2)

209-215: Missing assertion after polling for 200.

This poll_expected_status call returns a response, but unlike other assertions in this file (lines 72-75, 129-132, 164-167, 227-230), there's no explicit assert verifying the status code. Add an assertion for consistency and clearer test failure messages.

♻️ Proposed fix
-            poll_expected_status(
+            response = poll_expected_status(
                 request_session_http=request_session_http,
                 model_url=model_url_tinyllama_premium,
                 headers=explicit_headers,
                 payload=payload,
                 expected_statuses={200},
             )
+            assert response.status_code == 200, (
+                f"Expected 200 with extra AuthPolicy, got {response.status_code}: "
+                f"{(response.text or '')[:200]}"
+            )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`
around lines 209 - 215, The call to poll_expected_status using
request_session_http, model_url_tinyllama_premium, explicit_headers, and payload
currently ignores the returned response; capture its return (e.g., response =
poll_expected_status(...)) and add an assertion checking the status code (e.g.,
assert response.status_code == 200 or assert response.status_code in {200}) to
match the other tests (see earlier uses of poll_expected_status).

168-169: Extra blank line.

PEP 8: Two blank lines between top-level definitions, one blank line between method-level logical sections. This is extraneous.

🧹 Proposed fix
             f"{(baseline_resp.text or '')[:200]}"
         )
-
 
         suffix = generate_random_name()
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`
around lines 168 - 169, Remove the extraneous blank line flagged by PEP8 in
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py:
delete the extra empty row so there are exactly two blank lines between
top-level definitions (or one blank line between logical sections inside
functions/tests); locate the extra blank line between the surrounding top-level
definitions or inside the affected test function and collapse it to the proper
single or double blank-line spacing to satisfy PEP8.
tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py (3)

146-166: Fixture restoration relies on accessing attributes of a deleted resource.

Line 150 accesses maas_subscription_tinyllama_free.namespace after the resource was deleted on line 120. While the Python object still holds the cached attribute values, this pattern is fragile. Consider capturing original_namespace alongside original_name and original_priority before deletion for clarity and defensive coding.

♻️ Proposed fix
         original_name = maas_subscription_tinyllama_free.name
+        original_namespace = maas_subscription_tinyllama_free.namespace
         original_priority = getattr(maas_subscription_tinyllama_free, "priority", 0)
 
         LOGGER.info("Deleting original subscription %s", original_name)

Then use original_namespace in the restoration block:

             with MaaSSubscription(
                 client=admin_client,
                 name=original_name,
-                namespace=maas_subscription_tinyllama_free.namespace,
+                namespace=original_namespace,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`
around lines 146 - 166, The restore block is referencing
maas_subscription_tinyllama_free.namespace after that resource was deleted;
capture the namespace into a local variable (e.g., original_namespace) alongside
original_name and original_priority before deletion and then use that captured
original_namespace when constructing the MaaSSubscription in the finally block
(referencing maas_subscription_tinyllama_free, original_name, original_priority,
and restored_subscription to locate the code).

100-100: Remove commented-out fixture reference.

Dead code. Remove to reduce noise.

🧹 Proposed fix
         maas_free_group: str,
         maas_model_tinyllama_free,
-        # maas_auth_policy_tinyllama_free,
         model_url_tinyllama_free: str,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`
at line 100, Remove the dead commented-out fixture reference
"maas_auth_policy_tinyllama_free" from the test setup in
test_cascade_deletion.py; locate the commented line (e.g., "#
maas_auth_policy_tinyllama_free,") in the fixture list or parameterization and
delete it so the test file no longer contains the unused commented-out fixture
reference.

48-48: Remove commented-out code.

Dead code artifact. Either use maas_subscription_tinyllama_free for something meaningful or remove the comment entirely.

🧹 Proposed fix
-        # _ = maas_subscription_tinyllama_free
-
         with create_maas_subscription(
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`
at line 48, Remove the commented-out dead code line referencing
maas_subscription_tinyllama_free in test_cascade_deletion.py; either delete the
comment entirely or replace it with a meaningful assertion or fixture usage that
exercises maas_subscription_tinyllama_free (e.g., assign it to a variable or
call a helper/assertion). Locate the commented line containing "_ =
maas_subscription_tinyllama_free" and remove it if unused, or implement the
intended use of the maas_subscription_tinyllama_free fixture/function within the
test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`:
- Around line 116-120: The test deletes the class-scoped fixture
maas_subscription_tinyllama_free by calling
maas_subscription_tinyllama_free.clean_up(wait=True) and if an exception occurs
before the restoration block the fixture remains deleted and other tests fail;
wrap the deletion and restoration with explicit try/except/finally around the
clean_up and restoration logic in TestCascadeDeletion so any exception during
deletion is caught and logged (use LOGGER.error with exception info) and ensure
the restoration step for maas_subscription_tinyllama_free is executed in the
finally block with its own try/except that logs failures to restore (so failures
to restore are visible but don’t silently break subsequent tests).

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`:
- Line 217: The test double-deletes extra_auth_policy because it was created via
the context manager with teardown=True (extra_auth_policy) and then manually
calls extra_auth_policy.delete(wait=True); remove the explicit delete() call so
the context manager's __exit__ handles cleanup, or alternatively create
extra_auth_policy with teardown=False if you intend to delete manually—update
the code around the creation/usage of extra_auth_policy to use one cleanup
strategy consistently.

---

Nitpick comments:
In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`:
- Around line 146-166: The restore block is referencing
maas_subscription_tinyllama_free.namespace after that resource was deleted;
capture the namespace into a local variable (e.g., original_namespace) alongside
original_name and original_priority before deletion and then use that captured
original_namespace when constructing the MaaSSubscription in the finally block
(referencing maas_subscription_tinyllama_free, original_name, original_priority,
and restored_subscription to locate the code).
- Line 100: Remove the dead commented-out fixture reference
"maas_auth_policy_tinyllama_free" from the test setup in
test_cascade_deletion.py; locate the commented line (e.g., "#
maas_auth_policy_tinyllama_free,") in the fixture list or parameterization and
delete it so the test file no longer contains the unused commented-out fixture
reference.
- Line 48: Remove the commented-out dead code line referencing
maas_subscription_tinyllama_free in test_cascade_deletion.py; either delete the
comment entirely or replace it with a meaningful assertion or fixture usage that
exercises maas_subscription_tinyllama_free (e.g., assign it to a variable or
call a helper/assertion). Locate the commented line containing "_ =
maas_subscription_tinyllama_free" and remove it if unused, or implement the
intended use of the maas_subscription_tinyllama_free fixture/function within the
test.

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`:
- Around line 209-215: The call to poll_expected_status using
request_session_http, model_url_tinyllama_premium, explicit_headers, and payload
currently ignores the returned response; capture its return (e.g., response =
poll_expected_status(...)) and add an assertion checking the status code (e.g.,
assert response.status_code == 200 or assert response.status_code in {200}) to
match the other tests (see earlier uses of poll_expected_status).
- Around line 168-169: Remove the extraneous blank line flagged by PEP8 in
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py:
delete the extra empty row so there are exactly two blank lines between
top-level definitions (or one blank line between logical sections inside
functions/tests); locate the extra blank line between the surrounding top-level
definitions or inside the affected test function and collapse it to the proper
single or double blank-line spacing to satisfy PEP8.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 4747bdeb-f6ce-413c-88c1-6fbde185b81d

📥 Commits

Reviewing files that changed from the base of the PR and between 5517962 and 0974290.

📒 Files selected for processing (2)
  • tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py
  • tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py (1)

208-214: Missing assertion after polling for 200.

Unlike test_two_auth_policies_or_logic_allows_access (lines 129-132), this test does not assert on the response after polling for status 200. Add an explicit assertion for consistency and clearer failure diagnostics if the intermediate state check fails.

♻️ Proposed fix
-            poll_expected_status(
+            response_200 = poll_expected_status(
                 request_session_http=request_session_http,
                 model_url=model_url_tinyllama_premium,
                 headers=explicit_headers,
                 payload=payload,
                 expected_statuses={200},
             )
+            assert response_200.status_code == 200, (
+                f"Expected 200 with extra AuthPolicy, got {response_200.status_code}: "
+                f"{(response_200.text or '')[:200]}"
+            )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`
around lines 208 - 214, The poll_expected_status call in
test_multiple_auth_policies_per_model does not assert the actual response after
it returns 200; update the test to capture the return value from
poll_expected_status (e.g., response = poll_expected_status(...)) and add an
explicit assertion that response.status_code == 200 (and optionally assert
response is not None or inspect response.json() for expected keys), so failures
produce clear diagnostics; reference the poll_expected_status invocation in the
test to locate and modify the call.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`:
- Around line 208-214: The poll_expected_status call in
test_multiple_auth_policies_per_model does not assert the actual response after
it returns 200; update the test to capture the return value from
poll_expected_status (e.g., response = poll_expected_status(...)) and add an
explicit assertion that response.status_code == 200 (and optionally assert
response is not None or inspect response.json() for expected keys), so failures
produce clear diagnostics; reference the poll_expected_status invocation in the
test to locate and modify the call.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 66e814fc-87f0-4925-b05b-da0edb4a168b

📥 Commits

Reviewing files that changed from the base of the PR and between 0974290 and e3b317e.

📒 Files selected for processing (2)
  • tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py
  • tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py (1)

208-214: Assert the pre-delete 200 state explicitly.

Line 208 currently drops the response object, which weakens diagnostics if this step regresses. Capture and assert it before deletion.

Proposed fix
-            poll_expected_status(
+            pre_delete_response = poll_expected_status(
                 request_session_http=request_session_http,
                 model_url=model_url_tinyllama_premium,
                 headers=explicit_headers,
                 payload=payload,
                 expected_statuses={200},
             )
+            assert pre_delete_response.status_code == 200, (
+                f"Expected 200 before deleting extra AuthPolicy, got {pre_delete_response.status_code}: "
+                f"{(pre_delete_response.text or '')[:200]}"
+            )

As per coding guidelines, "REVIEW PRIORITIES: 3. Bug-prone patterns and error handling gaps".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`
around lines 208 - 214, The call to poll_expected_status currently discards the
response making regressions hard to debug; assign its return to a variable
(e.g., resp = poll_expected_status(...)) when calling poll_expected_status with
request_session_http, model_url_tinyllama_premium, headers=explicit_headers,
payload and expected_statuses={200}, then explicitly assert the response
indicates success (for example check resp.status_code == 200 or resp.status in
your test harness) before proceeding to deletion so the pre-delete 200 state is
captured and reported.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In
`@tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py`:
- Around line 208-214: The call to poll_expected_status currently discards the
response making regressions hard to debug; assign its return to a variable
(e.g., resp = poll_expected_status(...)) when calling poll_expected_status with
request_session_http, model_url_tinyllama_premium, headers=explicit_headers,
payload and expected_statuses={200}, then explicitly assert the response
indicates success (for example check resp.status_code == 200 or resp.status in
your test harness) before proceeding to deletion so the pre-delete 200 state is
captured and reported.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 47d95756-82a9-4399-93c8-8e0d145d74f6

📥 Commits

Reviewing files that changed from the base of the PR and between e3b317e and 1304dee.

📒 Files selected for processing (1)
  • tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py

mwaykole
mwaykole previously approved these changes Mar 14, 2026
Comment thread tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py Outdated
Comment thread tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py Outdated
Comment thread tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py Outdated
Comment thread tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py Outdated
SB159 and others added 4 commits March 16, 2026 18:09
Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
for more information, see https://pre-commit.ci

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
@SB159 SB159 force-pushed the feature/test-multiple-auth-policies-per-model branch from 41bdadb to 1eaae51 Compare March 16, 2026 23:11
@SB159 SB159 requested a review from dbasunag March 16, 2026 23:11
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py (1)

94-95: ⚠️ Potential issue | 🟠 Major

Put the destructive delete inside the restoration guard.

maas_subscription_tinyllama_free.clean_up(wait=True) at Line 95 runs before try/finally; if it throws, restoration never runs. That can leave class-scoped state broken for subsequent tests.

Proposed fix
-        LOGGER.info("Deleting original subscription %s", original_name)
-        maas_subscription_tinyllama_free.clean_up(wait=True)
-
         payload = chat_payload_for_url(model_url=model_url_tinyllama_free)
 
         try:
+            LOGGER.info("Deleting original subscription %s", original_name)
+            maas_subscription_tinyllama_free.clean_up(wait=True)
+
             response = poll_expected_status(
                 request_session_http=request_session_http,
                 model_url=model_url_tinyllama_free,
@@
-        finally:
-            with MaaSSubscription(
-                client=admin_client,
-                name=original_name,
-                namespace=maas_subscription_tinyllama_free.namespace,
-                owner={
-                    "groups": [{"name": maas_free_group}],
-                },
-                model_refs=[
-                    {
-                        "name": maas_model_tinyllama_free.name,
-                        "namespace": maas_model_tinyllama_free.namespace,
-                        "tokenRateLimits": [{"limit": 100, "window": "1m"}],
-                    }
-                ],
-                priority=original_priority,
-                teardown=False,
-                wait_for_resource=True,
-            ) as restored_subscription:
-                restored_subscription.wait_for_condition(
-                    condition="Ready",
-                    status="True",
-                    timeout=300,
-                )
-                LOGGER.info("Restored original subscription %s", restored_subscription.name)
+        finally:
+            try:
+                with MaaSSubscription(
+                    client=admin_client,
+                    name=original_name,
+                    namespace=maas_subscription_tinyllama_free.namespace,
+                    owner={
+                        "groups": [{"name": maas_free_group}],
+                    },
+                    model_refs=[
+                        {
+                            "name": maas_model_tinyllama_free.name,
+                            "namespace": maas_model_tinyllama_free.namespace,
+                            "tokenRateLimits": [{"limit": 100, "window": "1m"}],
+                        }
+                    ],
+                    priority=original_priority,
+                    teardown=False,
+                    wait_for_resource=True,
+                ) as restored_subscription:
+                    restored_subscription.wait_for_condition(
+                        condition="Ready",
+                        status="True",
+                        timeout=300,
+                    )
+                    LOGGER.info("Restored original subscription %s", restored_subscription.name)
+            except Exception:
+                LOGGER.exception("Failed restoring original subscription %s", original_name)
+                raise

As per coding guidelines, **: REVIEW PRIORITIES: 3. Bug-prone patterns and error handling gaps.

Also applies to: 99-145

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`
around lines 94 - 95, The call to
maas_subscription_tinyllama_free.clean_up(wait=True) is executed outside the
restoration guard and can raise, preventing the finally/restoration code from
running; move the destructive delete into the try/finally block (i.e., perform
maas_subscription_tinyllama_free.clean_up(wait=True) inside the try or inside
the finally after any safe checks) so that the restoration/teardown code always
executes, and ensure any other destructive cleanup calls in the same test (lines
~99-145) follow the same pattern and reference the same objects
(maas_subscription_tinyllama_free, original_name) to keep class-scoped state
consistent for subsequent tests.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/model_serving/maas_billing/maas_subscription/conftest.py`:
- Around line 509-558: The setup creates MaaSAuthPolicy (extra_auth_policy) and
a subscription (system_authenticated_subscription) and then calls
wait_for_condition; wrap the post-creation logic in a try/finally so cleanup
always runs on failure: after creating extra_auth_policy and
system_authenticated_subscription (via MaaSAuthPolicy and
create_maas_subscription) enter try: call extra_auth_policy.wait_for_condition
and system_authenticated_subscription.wait_for_condition and yield the dict, and
in finally: if extra_auth_policy.exists: call
extra_auth_policy.clean_up(wait=True) (and similarly clean up
system_authenticated_subscription if needed), ensuring you reference the
existing symbols extra_auth_policy, system_authenticated_subscription,
wait_for_condition, and clean_up so resources are removed even when waits throw.
- Around line 466-493: The fixture's cleanup must be idempotent and always run:
wrap the setup/yield logic around create_maas_subscription and
temporary_subscription.wait_for_condition in a try/finally so cleanup runs even
if setup fails, and before calling temporary_subscription.clean_up(check) verify
the resource still exists by calling temporary_subscription.exists(); only call
clean_up(wait=True) when exists() returns True (mirroring the pattern used
elsewhere in this file). Ensure you keep the yield of temporary_subscription
inside the try so finally always executes and do not remove the teardown=False
argument to create_maas_subscription.

---

Duplicate comments:
In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`:
- Around line 94-95: The call to
maas_subscription_tinyllama_free.clean_up(wait=True) is executed outside the
restoration guard and can raise, preventing the finally/restoration code from
running; move the destructive delete into the try/finally block (i.e., perform
maas_subscription_tinyllama_free.clean_up(wait=True) inside the try or inside
the finally after any safe checks) so that the restoration/teardown code always
executes, and ensure any other destructive cleanup calls in the same test (lines
~99-145) follow the same pattern and reference the same objects
(maas_subscription_tinyllama_free, original_name) to keep class-scoped state
consistent for subsequent tests.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 6fbf85f0-18e5-4aaa-a93b-217cf0b5ae0a

📥 Commits

Reviewing files that changed from the base of the PR and between 1304dee and 41bdadb.

📒 Files selected for processing (3)
  • tests/model_serving/maas_billing/maas_subscription/conftest.py
  • tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py
  • tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py

Comment thread tests/model_serving/maas_billing/maas_subscription/conftest.py
Comment thread tests/model_serving/maas_billing/maas_subscription/conftest.py
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py (1)

91-95: Mutating class-scoped fixture risks test isolation.

Deleting maas_subscription_tinyllama_free (class-scoped, line 95) affects other tests in TestCascadeDeletion that depend on it. If restoration in finally fails, subsequent tests will fail with misleading errors.

Consider wrapping the restoration in its own try/except to log failures explicitly:

Proposed fix
         finally:
+            try:
                 with MaaSSubscription(
                     client=admin_client,
                     name=original_name,
                     namespace=maas_subscription_tinyllama_free.namespace,
                     owner={
                         "groups": [{"name": maas_free_group}],
                     },
                     model_refs=[
                         {
                             "name": maas_model_tinyllama_free.name,
                             "namespace": maas_model_tinyllama_free.namespace,
                             "tokenRateLimits": [{"limit": 100, "window": "1m"}],
                         }
                     ],
                     priority=original_priority,
                     teardown=False,
                     wait_for_resource=True,
                 ) as restored_subscription:
                     restored_subscription.wait_for_condition(
                         condition="Ready",
                         status="True",
                         timeout=300,
                     )
                     LOGGER.info("Restored original subscription %s", restored_subscription.name)
+            except Exception as restore_err:
+                LOGGER.error(
+                    "Failed to restore subscription %s: %s. Subsequent tests may fail.",
+                    original_name,
+                    restore_err,
+                )
+                raise
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`
around lines 91 - 95, The test mutates the class-scoped fixture
maas_subscription_tinyllama_free by calling
maas_subscription_tinyllama_free.clean_up(wait=True), which can break other
tests in TestCascadeDeletion; fix by creating and deleting a temporary
subscription (or clone) for this test instead of operating on the class fixture,
and if you must restore the fixture ensure the restoration is wrapped in its own
try/except that catches exceptions from the restore logic and logs them (use
LOGGER.error) so failures during restore do not propagate and break subsequent
tests; refer to maas_subscription_tinyllama_free, clean_up(wait=True),
original_name/original_priority and TestCascadeDeletion when making the changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py`:
- Around line 91-95: The test mutates the class-scoped fixture
maas_subscription_tinyllama_free by calling
maas_subscription_tinyllama_free.clean_up(wait=True), which can break other
tests in TestCascadeDeletion; fix by creating and deleting a temporary
subscription (or clone) for this test instead of operating on the class fixture,
and if you must restore the fixture ensure the restoration is wrapped in its own
try/except that catches exceptions from the restore logic and logs them (use
LOGGER.error) so failures during restore do not propagate and break subsequent
tests; refer to maas_subscription_tinyllama_free, clean_up(wait=True),
original_name/original_priority and TestCascadeDeletion when making the changes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 3dcfa3ad-af12-44c4-8cdc-de93f72c883c

📥 Commits

Reviewing files that changed from the base of the PR and between 41bdadb and 1eaae51.

📒 Files selected for processing (3)
  • tests/model_serving/maas_billing/maas_subscription/conftest.py
  • tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py
  • tests/model_serving/maas_billing/maas_subscription/test_multiple_auth_policies_per_model.py

Comment thread tests/model_serving/maas_billing/maas_subscription/conftest.py
Comment thread tests/model_serving/maas_billing/maas_subscription/test_cascade_deletion.py Outdated
Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
@dbasunag dbasunag merged commit 6fe4677 into opendatahub-io:main Mar 17, 2026
10 checks passed
@github-actions
Copy link
Copy Markdown

Status of building tag latest: success.
Status of pushing tag latest to image registry: success.

ssaleem-rh pushed a commit to ssaleem-rh/opendatahub-tests that referenced this pull request Mar 23, 2026
…ndatahub-io#1220)

* tests(maas): add multiple auth policy and cascade deletion tests

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>

* tests(maas): fix address coderabit review comments

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>

* address review comments

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>

* address review comments

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>

---------

Signed-off-by: Swati Mukund Bagal <sbagal@redhat.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Debarati Basu-Nag <dbasunag@redhat.com>
Signed-off-by: Shehan Saleem <ssaleem@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants