tests: Adjustments to tests, which frequently fail on downstream instances#944
tests: Adjustments to tests, which frequently fail on downstream instances#944matejnesuta wants to merge 2 commits intoguacsec:mainfrom
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideAdjusts end-to-end UI tests for advisory and vulnerability explorers to be more stable and aligned with the current DOM and data variability by using more specific selectors and removing a brittle assertion on SBOM total count. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- Using the
td[data-label="ID"]locator makes the test more specific but also more coupled to the table markup; consider centralizing this selector in a helper so future table structure changes require updates in only one place. - Dropping the assertion that the SBOMs table total results is 1 may reduce the strength of the test; if the downstream flakiness is due to additional rows, consider relaxing the condition (e.g., at least 1 matching row) instead of removing the check entirely.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Using the `td[data-label="ID"]` locator makes the test more specific but also more coupled to the table markup; consider centralizing this selector in a helper so future table structure changes require updates in only one place.
- Dropping the assertion that the SBOMs table total results is 1 may reduce the strength of the test; if the downstream flakiness is due to additional rows, consider relaxing the condition (e.g., at least 1 matching row) instead of removing the check entirely.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #944 +/- ##
==========================================
- Coverage 62.06% 61.27% -0.80%
==========================================
Files 207 207
Lines 3664 3664
Branches 831 831
==========================================
- Hits 2274 2245 -29
- Misses 1079 1115 +36
+ Partials 311 304 -7 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mrrajan
left a comment
There was a problem hiding this comment.
@matejnesuta Thanks for the PR, Added suggestion below:
| When Tab "Related SBOMs" is visible | ||
| Then The page title is "<vulnerabilityID>" | ||
| Then The SBOMs table is sorted by "Name" | ||
| Then The SBOMs table total results is 1 |
There was a problem hiding this comment.
I feel this step should work unless there is a additional SBOM affected by the vulnerability. I would suggest to verify if there are any additional sbom related to vulnerability and make use of table assertion method toHaveNumberOfRows to verify the expected row counts on the table.
There was a problem hiding this comment.
Hello. I removed the line, because when importers are enabled, there is usually a big chance that some additional Quarkus SBOM gets loaded into the instance, which causes the test to fail on this line. The very next line already assumes that the correct SBOM should exist and I felt like this line only adds additional constraints without much benefit.
Summary by Sourcery
Update flaky end-to-end tests for advisory and vulnerability explorers to match the current UI and reduce brittleness.
Tests: