Skip to content

[6.18.z] adjusted asserts for mcp user tests - #20867

Merged
Satellite-QE merged 1 commit into
6.18.zfrom
cherry-pick-6.18.z-7777535ba140615e9c13447e9f61c8b53e1f2f59
Feb 25, 2026
Merged

[6.18.z] adjusted asserts for mcp user tests#20867
Satellite-QE merged 1 commit into
6.18.zfrom
cherry-pick-6.18.z-7777535ba140615e9c13447e9f61c8b53e1f2f59

Conversation

@Satellite-QE

@Satellite-QE Satellite-QE commented Feb 25, 2026

Copy link
Copy Markdown
Collaborator

Cherrypick of PR: #20852

Problem Statement

targeting an issue that isn't reproducible outside of our regular automation runs, therefore a bit mysterious. My hypothesis is that on the first run attempt the mcp client queries the mcp server before the connection with satellite is established using custom credentials.

Solution

Allow for a second try if the aforementioned issue happens

Related Issues

Summary by Sourcery

Adjust mcp user permission system tests to better handle transient API connectivity issues when querying allowed resources.

Tests:

  • Retry the Foreman API call in the mcp user view permissions test when a transient max-retries error occurs before asserting success.
  • Strengthen negative permission assertions by verifying the specific access-denied error message in the mcp user view permissions test.

@Satellite-QE Satellite-QE added 6.18.z Introduced in or relating directly to Satellite 6.18 Auto_Cherry_Picked Automatically cherrypicked PR using GHA No-CherryPick PR doesnt need CherryPick to previous branches labels Feb 25, 2026
@Satellite-QE

Copy link
Copy Markdown
Collaborator Author

trigger: test-robottelo
pytest: tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions

@sourcery-ai

sourcery-ai Bot commented Feb 25, 2026

Copy link
Copy Markdown
Contributor
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adjusts the MCP user permissions system test to tolerate a transient MCP client connection issue by retrying the allowed-resource API call once on a specific retry error, and strengthens the assertion for the denied-resource path to check the exact access-denied error message.

File-Level Changes

Change Details Files
Add defensive retry logic for the allowed-resource MCP API call and tighten assertions for the denied-resource error response in the MCP user permissions test.
  • After the first MCP API call for the allowed resource, detect the specific transient failure case where the response data contains an 'error' field with a 'Max retries exceeded' message.
  • On detecting that transient error, immediately reissue the same MCP API call to give the client a second chance once the connection is properly established.
  • Keep the existing success assertion for the allowed resource, ensuring the retried call still must return the expected success message.
  • Extend the denied-resource assertions to also verify that the nested response error message is exactly 'Access denied', in addition to the existing check for the failure message substring.
tests/foreman/sys/test_mcp.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Consider extracting the transient Max retries exceeded retry logic into a small helper or fixture so it can be reused and tuned consistently across tests that call the Foreman API.
  • The new strict equality assertion on result.data['response']['error']['message'] == 'Access denied' may be brittle if upstream error messages change slightly; if stability becomes an issue, you might prefer a containment or pattern-based check instead.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider extracting the transient `Max retries exceeded` retry logic into a small helper or fixture so it can be reused and tuned consistently across tests that call the Foreman API.
- The new strict equality assertion on `result.data['response']['error']['message'] == 'Access denied'` may be brittle if upstream error messages change slightly; if stability becomes an issue, you might prefer a containment or pattern-based check instead.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@pondrejk pondrejk added the AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing label Feb 25, 2026
@pondrejk

Copy link
Copy Markdown
Contributor

trigger: test-robottelo
pytest: tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions

@Satellite-QE

Copy link
Copy Markdown
Collaborator Author

PRT Result

Build Number: 14469
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/sys/test_mcp.py::test_positive_mcp_user_view_permissions --external-logging
Test Result : ================== 4 passed, 43 warnings in 801.99s (0:13:21) ==================

@Satellite-QE Satellite-QE added the PRT-Passed Indicates that latest PRT run is passed for the PR label Feb 25, 2026
@Satellite-QE
Satellite-QE merged commit 17c5634 into 6.18.z Feb 25, 2026
19 of 20 checks passed
@Satellite-QE
Satellite-QE deleted the cherry-pick-6.18.z-7777535ba140615e9c13447e9f61c8b53e1f2f59 branch February 25, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.18.z Introduced in or relating directly to Satellite 6.18 Auto_Cherry_Picked Automatically cherrypicked PR using GHA AutoMerge_Cherry_Picked The cherrypicked PRs of master PR would be automerged if all checks passing No-CherryPick PR doesnt need CherryPick to previous branches PRT-Passed Indicates that latest PRT run is passed for the PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants