Skip to content

chore: skip flaky tests #17675

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 13, 2025
Merged

chore: skip flaky tests #17675

merged 5 commits into from
May 13, 2025

Conversation

chainchad
chainchad previously approved these changes May 12, 2025
Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and fe9a17b (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

12 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 66.6667% false false false 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 46s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestMCMSSignFireDrillChangeset 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 0s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.706666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_enabled 66.6667% false false true 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 44.525s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.993333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 45.126666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_enabled 66.6667% false false true 3 2 1 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 43.445s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestValidate 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.945s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 45.1s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m15.52s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 59.19s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.466666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

@erikburt erikburt force-pushed the chore/skip-flaky-tests-may-12 branch from fe81696 to c0da446 Compare May 12, 2025 23:35
@erikburt erikburt requested review from krehermann and chainchad May 12, 2025 23:35
Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and c0da446 (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

9 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 55.225s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 33.3333% false false true 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 45.85s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 33.3333% false false true 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 1m12.15s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSVariants/MCMS_disabled 33.3333% false false false 3 1 2 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 46.746666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestValidate 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.333333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 46.69s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.993333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m23.14s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m16.76s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

krehermann
krehermann previously approved these changes May 13, 2025
jmank88
jmank88 previously approved these changes May 13, 2025
ajaskolski
ajaskolski previously approved these changes May 13, 2025
@erikburt erikburt dismissed stale reviews from ajaskolski, jmank88, and krehermann via 14f712c May 13, 2025 18:37
@cl-sonarqube-production
Copy link

Copy link
Contributor

Flakeguard Summary

Ran new or updated tests between develop and c2524dc (chore/skip-flaky-tests-may-12).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

8 Results
Name Pass Ratio Panicked? Timed Out? Race? Runs Successes Failures Skips Package Package Panicked? Avg Duration Code Owners
TestDeployMCMSWithTimelockV2 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 53.745s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestMCMSSignFireDrillChangeset 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 57.36s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2EVM/MCMS_disabled 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.976666666s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestSetConfigMCMSV2Solana/MCMS_disabled 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset false 52.17s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_Apply 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 50.133333333s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestFundMCMSignersChangeset_VerifyPreconditions 0% false false false 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m13.1s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestTransferToMCMSToTimelockSolana 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 59.25s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team
TestUpdateTimelockDelaySolana_VerifyPreconditions 0% false false true 3 0 3 0 github.com/smartcontractkit/chainlink/deployment/common/changeset/solana false 1m8.15s @smartcontractkit/ccip-tooling, @smartcontractkit/ccip-offchain, @smartcontractkit/keystone, @smartcontractkit/core, @smartcontractkit/deployment-automation, @smartcontractkit/cld-team

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

@erikburt erikburt requested a review from krehermann May 13, 2025 19:59
@krehermann krehermann added this pull request to the merge queue May 13, 2025
Merged via the queue into develop with commit 664ac91 May 13, 2025
133 of 143 checks passed
@krehermann krehermann deleted the chore/skip-flaky-tests-may-12 branch May 13, 2025 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants