SP-000 add path-suffixes to filter better for frontend E2E and vitests failing tests

[soemo]

Type of Change

• Bugfix
• Enhancement / new feature
• Refactoring
• Documentation

Description

See Slack

As we have in the frontend and studio repository different kinds of tests running in the CI, it is not so easy to find a prefix for all of them to list the failing/flaky tests on the default branch.

Therefore @ikortelainen came up with the idea to use a path suffix.

This is implemented in the PR

How to test

• code check
• run it locally with python3 find_flaky_tests.py --auth-token "**********" --prefix '' --path-suffixes '.spec.ts,.spec.tsx, .test.ts, .test.tsx' Staffbase/frontend master
• or check my results / diffs with some local extra logging

> python3 find_flaky_tests.py --auth-token "**********" --prefix '' --path-suffixes '.spec.ts,.spec.tsx, .test.ts, .test.tsx' Staffbase/frontend master

AppState(auth_token='****', org='Staffbase', repo='frontend', branch='master', slack_channel=None, path_suffixes=('.spec.ts', '.spec.tsx', '.test.ts', 'test.tsx'), prefix='', since=datetime.datetime(2025, 6, 1, 12, 42, 18, 891392), until=datetime.datetime(2025, 6, 8, 12, 42, 18, 891397))

in suffixes  test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/StaticPage.ts skipping
in suffixes  test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/StaticPage.ts skipping
in suffixes  test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/page-widgets/StaticContentWidget.ts skipping
in suffixes  test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/StaticPage.ts skipping

diff for two runs with and without the suffixes

diff find_flaky_tests_frontend.txt find_flaky_tests_frontend_only_prefix.txt
11a12,28
> test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/StaticPage.ts:
>     in 5900ec067af273fcd6a7ce00d1cccb8f50551fb8 on 2025-06-06 14:57:46+00:00, see https://github.com/Staffbase/frontend/actions/runs/15511684638/job/43673445290
..
> test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/page-widgets/StaticContentWidget.ts:
>     in 5900ec067af273fcd6a7ce00d1cccb8f50551fb8 on 2025-06-06 14:57:46+00:00, see https://github.com/Staffbase/frontend/actions/runs/15511684638/job/43673445290
..
> test/e2e-playwright/tests/api/chat-api.ts:
>     in 1bb06ecaefaf9e831db186f5a387fed8131267a7 on 2025-06-06 14:57:46+00:00, see https://github.com/Staffbase/frontend/actions/runs/15490497483/job/43614695140
...
> test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/BaseGeneral.ts:
>     in 1bb06ecaefaf9e831db186f5a387fed8131267a7 on 2025-06-06 14:57:46+00:00, see https://github.com/Staffbase/frontend/actions/runs/15490118050/job/43613499968
...
> test/e2e-playwright/tests/archived-webdriver-tests/pageobjects/Dialog.page.ts:
>     in bfb4c3cdb28679094aca8552e3ed747530755dce on 2025-06-05 10:06:11+00:00, see https://github.com/Staffbase/frontend/actions/runs/15437949376/job/43448679464

TODOs after PR merge

• add it to the gha-workflows
• use it int the frontend and studio

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[soemo]

I have read the CLA Document and I hereby sign the CLA

[Copilot]

Pull Request Overview

This PR adds an optional file path suffix filter to the find_flaky_tests.py script and updates the corresponding GitHub Action and documentation.

• Adds a new path_suffixes field in AppState, CLI parsing (--path-suffixes), and filters annotations by suffix in list_occurrences.
• Updates action.yml to expose the path-suffixes input to the GitHub Action.
• Extends README.md with an example showing how to specify path suffixes.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
find_flaky_tests.py	Introduces path_suffixes parsing and filtering logic.
action.yml	Defines path-suffixes input and passes it to the script.
README.md	Adds path-suffixes to the usage example under inputs.

Comments suppressed due to low confidence (2)

find_flaky_tests.py:60

• Add unit tests for parse_suffixes_tuple to verify correct parsing of comma-separated suffix strings and edge cases.

def parse_suffixes_tuple(s: str | None) -> tuple | None:

find_flaky_tests.py:236

• Add integration tests for list_occurrences to ensure annotations are correctly filtered by the provided suffixes.

                if state.path_suffixes and not ann.path.endswith(tuple(state.path_suffixes)):

[MarvinZeising]

Nice, thanks!
Why not file-suffixes? Since we use it to filter the files and not the paths where those files can be found?

[soemo]

Nice, thanks! Why not file-suffixes? Since we use it to filter the files and not the paths where those files can be found?

I named it path because the GitHub annotation has it as path too.