Skip to content

tests: Adjustments to tests, which frequently fail on downstream instances#944

Open
matejnesuta wants to merge 2 commits intoguacsec:mainfrom
matejnesuta:ui-test-fixv2
Open

tests: Adjustments to tests, which frequently fail on downstream instances#944
matejnesuta wants to merge 2 commits intoguacsec:mainfrom
matejnesuta:ui-test-fixv2

Conversation

@matejnesuta
Copy link
Contributor

@matejnesuta matejnesuta commented Mar 4, 2026

Summary by Sourcery

Update flaky end-to-end tests for advisory and vulnerability explorers to match the current UI and reduce brittleness.

Tests:

  • Adjust advisory explorer test to target the advisory ID via a more specific table cell locator.
  • Relax vulnerability explorer scenario by removing a strict assertion on SBOMs table total results.

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Mar 4, 2026

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adjusts end-to-end UI tests for advisory and vulnerability explorers to be more stable and aligned with the current DOM and data variability by using more specific selectors and removing a brittle assertion on SBOM total count.

File-Level Changes

Change Details Files
Stabilize advisory explorer result assertion by using a more specific table cell locator for advisory IDs.
  • Replace generic gridcell role-based locator with a CSS locator targeting the ID column cells in the advisory results table
  • Keep visibility expectation but narrow the search scope to the ID column to reduce false positives and selector flakiness
e2e/tests/ui/features/@advisory-explorer/advisory-explorer.step.ts
Relax vulnerability explorer SBOM table test to avoid brittle assumptions about result counts.
  • Remove the assertion that the SBOMs table total results must equal 1 from the vulnerability details scenario
  • Keep verification focused on the Name column contents rather than the exact number of SBOMs returned
e2e/tests/ui/features/@vulnerability-explorer/vulnerability-explorer.feature

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Using the td[data-label="ID"] locator makes the test more specific but also more coupled to the table markup; consider centralizing this selector in a helper so future table structure changes require updates in only one place.
  • Dropping the assertion that the SBOMs table total results is 1 may reduce the strength of the test; if the downstream flakiness is due to additional rows, consider relaxing the condition (e.g., at least 1 matching row) instead of removing the check entirely.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Using the `td[data-label="ID"]` locator makes the test more specific but also more coupled to the table markup; consider centralizing this selector in a helper so future table structure changes require updates in only one place.
- Dropping the assertion that the SBOMs table total results is 1 may reduce the strength of the test; if the downstream flakiness is due to additional rows, consider relaxing the condition (e.g., at least 1 matching row) instead of removing the check entirely.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@codecov
Copy link

codecov bot commented Mar 4, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.27%. Comparing base (bbe7025) to head (ec897d7).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #944      +/-   ##
==========================================
- Coverage   62.06%   61.27%   -0.80%     
==========================================
  Files         207      207              
  Lines        3664     3664              
  Branches      831      831              
==========================================
- Hits         2274     2245      -29     
- Misses       1079     1115      +36     
+ Partials      311      304       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@matejnesuta matejnesuta requested review from mrrajan and vobratil March 4, 2026 16:32
Copy link
Contributor

@mrrajan mrrajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matejnesuta Thanks for the PR, Added suggestion below:

When Tab "Related SBOMs" is visible
Then The page title is "<vulnerabilityID>"
Then The SBOMs table is sorted by "Name"
Then The SBOMs table total results is 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this step should work unless there is a additional SBOM affected by the vulnerability. I would suggest to verify if there are any additional sbom related to vulnerability and make use of table assertion method toHaveNumberOfRows to verify the expected row counts on the table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello. I removed the line, because when importers are enabled, there is usually a big chance that some additional Quarkus SBOM gets loaded into the instance, which causes the test to fail on this line. The very next line already assumes that the correct SBOM should exist and I felt like this line only adds additional constraints without much benefit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants