Skip to content

[6.19.z] adjusted asserts for mcp user tests - #20868

Merged
lpramuk merged 1 commit into
6.19.zfrom
cherry-pick-6.19.z-7777535ba140615e9c13447e9f61c8b53e1f2f59
Feb 25, 2026
Merged

[6.19.z] adjusted asserts for mcp user tests#20868
lpramuk merged 1 commit into
6.19.zfrom
cherry-pick-6.19.z-7777535ba140615e9c13447e9f61c8b53e1f2f59

Conversation

@Satellite-QE

@Satellite-QE Satellite-QE commented Feb 25, 2026

Copy link
Copy Markdown
Collaborator

Cherrypick of PR: #20852

Problem Statement

targeting an issue that isn't reproducible outside of our regular automation runs, therefore a bit mysterious. My hypothesis is that on the first run attempt the mcp client queries the mcp server before the connection with satellite is established using custom credentials.

Solution

Allow for a second try if the aforementioned issue happens

Related Issues

Summary by Sourcery

Relax flaky MCP user view permissions test behavior around transient connection failures and strengthen negative access assertion.

Tests:

  • Add a retry in the MCP user view permissions test when a transient 'Max retries exceeded' error is returned from the API call.
  • Assert the MCP client returns an 'Access denied' error message for disallowed resources in the negative permission path.

@Satellite-QE Satellite-QE added 6.19.z Auto_Cherry_Picked Automatically cherrypicked PR using GHA No-CherryPick PR doesnt need CherryPick to previous branches labels Feb 25, 2026
@Satellite-QE

Copy link
Copy Markdown
Collaborator Author

trigger: test-robottelo
pytest: tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions

@sourcery-ai

sourcery-ai Bot commented Feb 25, 2026

Copy link
Copy Markdown
Contributor
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adjusts the MCP user permissions system test to tolerate a transient connection error by retrying once and strengthens the negative-permission assertion to validate the exact access-denied error payload.

File-Level Changes

Change Details Files
Make the MCP user view-permissions test resilient to a transient "Max retries exceeded" error and tighten the assertion for denied resource access.
  • After the initial MCP client call to the Foreman API, detect a specific transient error condition in the result payload and perform a single retry of the same request.
  • After verifying that access to the denied resource fails, add an assertion that the response payload contains the expected "Access denied" error message in the nested error structure.
tests/foreman/sys/test_mcp.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The retry logic based on a specific error message string is quite brittle; consider checking for a more structured error signal (e.g., status code/exception type) or wrapping this in a small helper so similar transient cases can be handled consistently across tests.
  • The new assertion result.data['response']['error']['message'] assumes a deeply nested structure that may not always exist; you might want to guard this with safer access (e.g., get or an explicit assertion on the response shape) to avoid unexpected KeyErrors masking the test intent.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The retry logic based on a specific error message string is quite brittle; consider checking for a more structured error signal (e.g., status code/exception type) or wrapping this in a small helper so similar transient cases can be handled consistently across tests.
- The new assertion `result.data['response']['error']['message']` assumes a deeply nested structure that may not always exist; you might want to guard this with safer access (e.g., `get` or an explicit assertion on the response shape) to avoid unexpected KeyErrors masking the test intent.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@pondrejk

Copy link
Copy Markdown
Contributor

trigger: test-robottelo
pytest: tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions

@pondrejk pondrejk added the AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing label Feb 25, 2026
@Satellite-QE

Copy link
Copy Markdown
Collaborator Author

PRT Result

Build Number: 14468
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions --external-logging
Test Result : ================== 4 passed, 40 warnings in 828.89s (0:13:48) ==================

@Satellite-QE Satellite-QE added the PRT-Passed Indicates that latest PRT run is passed for the PR label Feb 25, 2026
@lpramuk
lpramuk merged commit 385d53d into 6.19.z Feb 25, 2026
18 of 20 checks passed
@lpramuk
lpramuk deleted the cherry-pick-6.19.z-7777535ba140615e9c13447e9f61c8b53e1f2f59 branch February 25, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.19.z Auto_Cherry_Picked Automatically cherrypicked PR using GHA AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing No-CherryPick PR doesnt need CherryPick to previous branches PRT-Passed Indicates that latest PRT run is passed for the PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants