Skip to content

chore: skip flaky tests #17675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
May 13, 2025
Merged

chore: skip flaky tests #17675

merged 5 commits into from
May 13, 2025

Conversation

erikburt
Copy link
Collaborator

@erikburt erikburt self-assigned this May 12, 2025
@erikburt erikburt requested review from a team as code owners May 12, 2025 17:06
@erikburt erikburt mentioned this pull request May 12, 2025
Closed
chainchad
chainchad previously approved these changes May 12, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and fe9a17b (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

12 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 66.6667% false false false 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 46s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestMCMSSignFireDrillChangeset 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 0s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.706666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_enabled 66.6667% false false true 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 44.525s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.993333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 45.126666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_enabled 66.6667% false false true 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 43.445s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestValidate 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.945s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 45.1s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m15.52s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 59.19s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.466666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

krehermann
krehermann requested changes May 12, 2025
View reviewed changes
@erikburt erikburt force-pushed the chore/skip-flaky-tests-may-12 branch from fe81696 to c0da446 Compare May 12, 2025 23:35
@erikburt erikburt requested review from krehermann and chainchad May 12, 2025 23:35
@github-actions GitHub Actions
Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and c0da446 (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

9 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 55.225s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 33.3333% false false true 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 45.85s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 33.3333% false false true 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 1m12.15s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 46.746666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestValidate 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.333333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 46.69s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.993333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m23.14s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m16.76s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

krehermann
krehermann previously approved these changes May 13, 2025
View reviewed changes
jmank88
jmank88 previously approved these changes May 13, 2025
View reviewed changes
ajaskolski
ajaskolski previously approved these changes May 13, 2025
View reviewed changes
@erikburt erikburt dismissed stale reviews from ajaskolski, jmank88, and krehermann via 14f712c May 13, 2025 18:37
@cl-sonarqube-production
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

@github-actions GitHub Actions
Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and c2524dc (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

8 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.745s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestMCMSSignFireDrillChangeset 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 57.36s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.976666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.17s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 50.133333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.1s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 59.25s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m8.15s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

@erikburt erikburt requested a review from krehermann May 13, 2025 19:59
@krehermann krehermann added this pull request to the merge queue May 13, 2025
Merged via the queue into develop with commit 664ac91 May 13, 2025
133 of 143 checks passed
@krehermann krehermann deleted the chore/skip-flaky-tests-may-12 branch May 13, 2025 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jmank88 jmank88 jmank88 approved these changes

@krehermann krehermann krehermann approved these changes

@chainchad chainchad chainchad approved these changes

@ajaskolski ajaskolski ajaskolski left review comments

@winder winder Awaiting requested review from winder winder is a code owner automatically assigned from smartcontractkit/ccip-offchain

@asoliman92 asoliman92 Awaiting requested review from asoliman92 asoliman92 is a code owner automatically assigned from smartcontractkit/ccip-offchain

@matYang matYang Awaiting requested review from matYang matYang is a code owner automatically assigned from smartcontractkit/ccip-offchain

@makramkd makramkd Awaiting requested review from makramkd makramkd is a code owner automatically assigned from smartcontractkit/ccip-offchain

@dimkouv dimkouv Awaiting requested review from dimkouv dimkouv is a code owner automatically assigned from smartcontractkit/ccip-offchain

@mateusz-sekara mateusz-sekara Awaiting requested review from mateusz-sekara mateusz-sekara is a code owner automatically assigned from smartcontractkit/ccip-offchain

@0xAustinWang 0xAustinWang Awaiting requested review from 0xAustinWang 0xAustinWang is a code owner automatically assigned from smartcontractkit/ccip-offchain

Assignees

@erikburt erikburt

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@erikburt @jmank88 @krehermann @ajaskolski @chainchad