-
Notifications
You must be signed in to change notification settings - Fork 27
tests: Add UI tests for the Search page #721
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
matejnesuta
wants to merge
30
commits into
guacsec:main
Choose a base branch
from
matejnesuta:search
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
658fb4d
added attachment export
153ff07
resolve linter
e981db1
tweaks
86f60d0
Merge branch 'main' into ci_traces
matejnesuta 1143e4a
this seems good
5baa9f8
linting and CI fixes
90ca7e8
attempt to fix a check flake
ebefa2c
Merge branch 'main' into search
matejnesuta c3b58a6
attempt to fix sorting flake
29b0c74
attempt #2
41a6ead
fix typo
115fbe1
rewrite gherkin parameter
d506a7b
Merge branch 'main' into search
matejnesuta 0d49f23
Merge branch 'main' into search
matejnesuta 88af1f5
:ghost: Bump the tools group with 3 updates (#691)
dependabot[bot] 80aed53
mfsdjklfdjkl
429f62f
fixed git conflicts
4f5b11f
deprecate old helpers
ff117b1
broken test fix
4a9ba0d
uncomment a test case
53dea50
remove fixed numbers from gherkin
a6c63e1
remove all changes to old helpers
92f16b8
move helper functions from search.step.ts
692dba2
remove redundant entry in a gherkin table
012fa6f
make sorting tests better
a0043cd
make gherkin more readable
1129c5b
linter fixes
3ac2d04
move stuff around
e93e5b4
moved helper functions
ea5eb21
move utils elsewhere
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,152 @@ | ||
| import { expect as baseExpect } from "@playwright/test"; | ||
| import type { SearchPage } from "../pages/search-page/SearchPage"; | ||
| import type { MatcherResult } from "./types"; | ||
|
|
||
| export interface SearchPageMatchers { | ||
| toHaveAutoFillHidden(): Promise<MatcherResult>; | ||
| toHaveRelevantAutoFillResults(searchText: string): Promise<MatcherResult>; | ||
| toHaveAutoFillCategoriesWithinLimit(limit: number): Promise<MatcherResult>; | ||
| } | ||
|
|
||
| type SearchPageMatcherDefinitions = { | ||
| readonly [K in keyof SearchPageMatchers]: ( | ||
| receiver: SearchPage, | ||
| ...args: Parameters<SearchPageMatchers[K]> | ||
| ) => Promise<MatcherResult>; | ||
| }; | ||
|
|
||
| export const searchPageAssertions = | ||
| baseExpect.extend<SearchPageMatcherDefinitions>({ | ||
| toHaveAutoFillHidden: async ( | ||
| searchPage: SearchPage, | ||
| ): Promise<MatcherResult> => { | ||
| try { | ||
| const menu = searchPage.getAutoFillMenu(); | ||
| await baseExpect(menu).not.toBeVisible(); | ||
|
|
||
| return { | ||
| pass: true, | ||
| message: () => "Autofill menu is not visible", | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| error instanceof Error ? error.message : String(error), | ||
| }; | ||
| } | ||
| }, | ||
|
|
||
| toHaveRelevantAutoFillResults: async ( | ||
| searchPage: SearchPage, | ||
| searchText: string, | ||
| ): Promise<MatcherResult> => { | ||
| try { | ||
| const menu = searchPage.getAutoFillMenu(); | ||
| await baseExpect(menu).toBeVisible(); | ||
|
|
||
| const menuItems = searchPage.getAutoFillMenuItems(); | ||
| const count = await menuItems.count(); | ||
|
|
||
| if (count === 0) { | ||
| return { | ||
| pass: false, | ||
| message: () => "Autofill menu has no items", | ||
| }; | ||
| } | ||
|
|
||
| // Check that at least one menu item contains the search text | ||
| const searchTextLower = searchText.toLowerCase(); | ||
| let foundMatch = false; | ||
|
|
||
| for (let i = 0; i < count; i++) { | ||
| const item = menuItems.nth(i); | ||
| const text = await item.textContent(); | ||
| if (text?.toLowerCase().includes(searchTextLower)) { | ||
| foundMatch = true; | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| if (!foundMatch) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| `No autofill items contain search text "${searchText}"`, | ||
| }; | ||
| } | ||
|
|
||
| return { | ||
| pass: true, | ||
| message: () => `Autofill has relevant results for "${searchText}"`, | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| error instanceof Error ? error.message : String(error), | ||
| }; | ||
| } | ||
| }, | ||
|
|
||
| toHaveAutoFillCategoriesWithinLimit: async ( | ||
| searchPage: SearchPage, | ||
| limit: number, | ||
| ): Promise<MatcherResult> => { | ||
| try { | ||
| const menuItems = searchPage.getAutoFillMenuLinks(); | ||
|
|
||
| const categoryCount: Record<string, number> = { | ||
| advisories: 0, | ||
| packages: 0, | ||
| sboms: 0, | ||
| vulnerabilities: 0, | ||
| }; | ||
|
|
||
| const count = await menuItems.count(); | ||
|
|
||
| for (let i = 0; i < count; i++) { | ||
| const link = menuItems.nth(i); | ||
| const href = await link.getAttribute("href"); | ||
|
|
||
| if (href?.includes("/advisories/")) { | ||
| categoryCount.advisories++; | ||
| } else if (href?.includes("/packages/")) { | ||
| categoryCount.packages++; | ||
| } else if (href?.includes("/sboms/")) { | ||
| categoryCount.sboms++; | ||
| } else if (href?.includes("/vulnerabilities/")) { | ||
| categoryCount.vulnerabilities++; | ||
| } | ||
| } | ||
|
|
||
| // Check if any category exceeds the limit | ||
| const violations: string[] = []; | ||
| for (const [category, count] of Object.entries(categoryCount)) { | ||
| if (count > limit) { | ||
| violations.push(`${category}: ${count} > ${limit}`); | ||
| } | ||
| } | ||
|
|
||
| if (violations.length > 0) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| `Categories exceed limit of ${limit}: ${violations.join(", ")}`, | ||
| }; | ||
| } | ||
|
|
||
| return { | ||
| pass: true, | ||
| message: () => | ||
| `All categories within limit of ${limit}: ${JSON.stringify(categoryCount)}`, | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| error instanceof Error ? error.message : String(error), | ||
| }; | ||
| } | ||
| }, | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| import { expect as baseExpect } from "@playwright/test"; | ||
| import type { SearchPageTabs } from "../pages/SearchPageTabs"; | ||
| import type { MatcherResult } from "./types"; | ||
|
|
||
| export interface SearchPageTabsMatchers { | ||
| toHaveTabCountAtLeast(minCount: number): Promise<MatcherResult>; | ||
| } | ||
|
|
||
| type SearchPageTabsMatcherDefinitions = { | ||
| readonly [K in keyof SearchPageTabsMatchers]: ( | ||
| receiver: SearchPageTabs, | ||
| ...args: Parameters<SearchPageTabsMatchers[K]> | ||
| ) => Promise<MatcherResult>; | ||
| }; | ||
|
|
||
| export const searchPageTabsAssertions = | ||
| baseExpect.extend<SearchPageTabsMatcherDefinitions>({ | ||
| toHaveTabCountAtLeast: async ( | ||
| searchPageTabs: SearchPageTabs, | ||
| minCount: number, | ||
| ): Promise<MatcherResult> => { | ||
| try { | ||
| const badge = searchPageTabs.getBadge(); | ||
|
|
||
| // Wait until the badge has some text | ||
| await baseExpect(badge).toHaveText(/[\d]/, { timeout: 60000 }); | ||
|
|
||
| const countText = await badge.textContent(); | ||
|
|
||
| // Remove anything that isn't a digit | ||
| const match = countText?.match(/\d+/); | ||
| if (!match) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| `Could not parse badge count from tab: got "${countText}"`, | ||
| }; | ||
| } | ||
|
|
||
| const count = parseInt(match[0], 10); | ||
|
|
||
| if (count < minCount) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| `Expected tab to have at least ${minCount} results, but got ${count}`, | ||
| }; | ||
| } | ||
|
|
||
| return { | ||
| pass: true, | ||
| message: () => `Tab has ${count} results (>= ${minCount})`, | ||
| }; | ||
| } catch (error) { | ||
| return { | ||
| pass: false, | ||
| message: () => | ||
| error instanceof Error ? error.message : String(error), | ||
| }; | ||
| } | ||
| }, | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| Feature: Search | ||
| As a Devsecops Engineer | ||
| I want to perform searching across vulnerabilities, SBOMs and packages, specific searches for CVE IDs, SBOM titles, package names and show results that are easy to navigate to the specific item of interest. | ||
|
|
||
| Background: | ||
| Given User is authenticated | ||
| And User is on the Search page | ||
|
|
||
| Scenario: User visits search page without filling anything | ||
| Then a total number of "SBOMs" should be visible in the tab | ||
| And a total number of "Packages" should be visible in the tab | ||
| And a total number of "Vulnerabilities" should be visible in the tab | ||
| And a total number of "Advisories" should be visible in the tab | ||
|
|
||
| Scenario Outline: User toggles the "<types>" list and manipulates the list | ||
| When User selects the Tab "<types>" | ||
| Then the "<types>" list should have the "<filters>" filter set | ||
| And the "<types>" list should be sortable | ||
matejnesuta marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| And the "<types>" list should be limited to 10 items | ||
| And the user should be able to switch to next "<types>" items | ||
| And the user should be able to increase pagination for the "<types>" | ||
matejnesuta marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| And First column on the search results should have the link to "<types>" explorer pages | ||
|
|
||
| Examples: | ||
| |types|filters| | ||
| |SBOMs|Created on| | ||
| |Packages|Type, Architecture| | ||
| |Vulnerabilities|CVSS, Published| | ||
| |Advisories|Revision| | ||
|
|
||
| Scenario Outline: Download Links on the "<types>" Search Result list | ||
| When User selects the Tab "<types>" | ||
| Then Tab "<types>" is visible | ||
| And Download link should be available for the "<types>" list | ||
|
|
||
| Examples: | ||
| |types| | ||
| |SBOMs| | ||
| |Advisories| | ||
|
|
||
| Scenario Outline: Autofill shows results matched on <input> | ||
| When user starts typing a "<input>" in the search bar | ||
| Then the autofill dropdown should display items matching the "<input>" | ||
| And the results should be limited to 5 suggestions | ||
|
|
||
| Examples: | ||
| |input| | ||
| |quarkus| | ||
| |CVE-2022| | ||
| |policies| | ||
|
|
||
| Scenario: Search bar should not preview anything when no matches are found | ||
| And user starts typing a "non-existent name" in the search bar | ||
| Then The autofill drop down should not show any values | ||
|
|
||
| Scenario Outline: User searches for a specific "<types>" | ||
| When user types a "<type-instance>" in the search bar | ||
| And user presses Enter | ||
| And User selects the Tab "<types>" | ||
| Then the "<types>" list should display the specific "<type-instance>" | ||
| And the list should be limited to 10 items or less | ||
| And the user should be able to filter "<types>" | ||
| And user clicks on the "<type-instance>" "<types>" link | ||
| And the user should be navigated to the specific "<type-instance>" page | ||
|
|
||
| Examples: | ||
| |types|type-instance| | ||
| |SBOMs|quarkus-bom| | ||
| |Vulnerabilities|CVE-2022-45787| | ||
| |Packages|quarkus| | ||
| |Advisories|CVE-2022-45787| | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.