Skip to content

chore: workflow to check for flaky e2e changes #20435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

anikdhabal
Copy link
Contributor

@anikdhabal anikdhabal commented Mar 28, 2025

What does this PR do?

Added a new workflow to check for flaky e2e changes inside a PR

This pull request introduces a new workflow to detect and report flaky end-to-end (E2E) tests. The most important changes include the creation of a new GitHub Actions workflow file and the integration of this workflow into the existing pull request workflow.

New workflow for flaky E2E tests:

  • .github/workflows/flaky-e2e-check.yml: Added a new workflow to detect flaky E2E tests. This workflow checks for changed E2E test files, runs them multiple times, calculates their flakiness, and uploads a report.

Integration into existing workflows:

  • .github/workflows/pr.yml: Integrated the new flaky E2E check workflow into the existing pull request workflow, ensuring it runs when specific conditions are met.

@graphite-app graphite-app bot requested a review from a team March 28, 2025 17:26
@keithwillcode keithwillcode added the core area: core, team members only label Mar 28, 2025
@dosubot dosubot bot added automated-tests area: unit tests, e2e tests, playwright ci area: CI, DX, pipeline, github actions labels Mar 28, 2025
@anikdhabal anikdhabal changed the title chore: chore: workflow to check for flaky e2e changes chore: workflow to check for flaky e2e changes Mar 28, 2025
Copy link

graphite-app bot commented Mar 28, 2025

Graphite Automations

"Add consumer team as reviewer" took an action on this PR • (03/28/25)

1 reviewer was added to this PR based on Keith Williams's automation.

Copy link

vercel bot commented Mar 28, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
cal ⬜️ Ignored (Inspect) Visit Preview Mar 28, 2025 5:35pm
calcom-web-canary ⬜️ Ignored (Inspect) Visit Preview Mar 28, 2025 5:35pm

@zomars
Copy link
Member

zomars commented Mar 28, 2025

How does this differ from the HTML reports we already generate?

Copy link
Member

@zomars zomars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of dedicating workflow jsut to detect flakyness. I would rather incorportae this into our actual testing runs and maybe ping on slack if a flakyness is detected.

@github-actions github-actions bot marked this pull request as draft March 28, 2025 18:16
Copy link
Contributor

This PR is being marked as stale due to inactivity.

@github-actions github-actions bot added the Stale label Apr 12, 2025
@github-actions github-actions bot removed the Stale label Apr 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automated-tests area: unit tests, e2e tests, playwright ci area: CI, DX, pipeline, github actions core area: core, team members only
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants