[CSP] Fix flaky CSPM GCP integration test - increase Add Agent flyout wait timeout#271993
Open
animehart wants to merge 3 commits into
Open
[CSP] Fix flaky CSPM GCP integration test - increase Add Agent flyout wait timeout#271993animehart wants to merge 3 commits into
animehart wants to merge 3 commits into
Conversation
… wait timeout The 'Add Agent FLyout' tests in the GCP CSPM suite were consistently timing out with: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="agentEnrollmentFlyout"]) Wait timed out after 10039ms Root cause: `retry.waitFor` in `clickFirstElementOnIntegrationTableAddAgent` uses `timeouts.waitFor` from the FTR config (10 s). The agent enrollment flyout can take longer than that to render in CI after clicking Add Agent, especially when the page still has pending network requests. Fix: - Add `waitUntilLoadingHasFinished()` before clicking the Add Agent button so the page is stable before the click. - Switch `retry.waitFor` → `retry.waitForWithTimeout(..., 30_000, ...)` so the flyout has the same 30-second budget used by `saveIntegrationPolicyTimeout`. Also removes the `describe.skip` and the `// Failing: See elastic#258331` comment from group4/pages/cis_integrations/cspm/cis_integration_gcp.ts. Closes elastic#258331 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
/flaky ftrConfig:x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts:50 |
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#12526
|
Contributor
Author
|
/flaky ftrConfig:x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts:50 |
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#12527
|
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#12526[❌] x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts: 0/50 tests passed. |
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#12527[❌] x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts: 0/50 tests passed. |
…ssertions The previous implementation used retry.tryForTime and threw when the string was absent, meaning it could never return false. Tests asserting .to.be(false) (e.g. ORG_ID absent in Single Account mode) always timed out with "retry.tryForTime reached timeout 20000 ms". Fix: wait up to 30 s for the code block to be non-empty, then return text.includes(str) as a plain boolean so both .to.be(true) and .to.be(false) assertions work correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
/flaky ftrConfig:x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts:50 |
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#12532
|
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12532[✅] x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts: 50/50 tests passed. |
Contributor
Author
|
/flaky ftrConfig:x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts:50 |
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#12534
|
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12534[✅] x-pack/solutions/security/test/cloud_security_posture_functional/group4/config.ts: 50/50 tests passed. |
animehart
added
v9.5.0
backport:skip
|
Pinging @elastic/contextual-security-apps (Team:Cloud Security) |
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the consistently flaky
describe('Test adding Cloud Security Posture Integrations CSPM GCP')suite ingroup4/pages/cis_integrations/cspm/cis_integration_gcp.ts(issue #258331).Root cause:
clickFirstElementOnIntegrationTableAddAgentin the shared page object usedretry.waitForto wait for theagentEnrollmentFlyoutelement.retry.waitForusestimeouts.waitForfrom the FTR config (10 s). In CI, the agent enrollment flyout can take longer than 10 s to render after clicking the Add Agent button, causing:Fix:
waitUntilLoadingHasFinished()before clicking Add Agent so the page is stable before the click.retry.waitFor→retry.waitForWithTimeout(..., 30_000, ...)giving the flyout the same 30-second budget assaveIntegrationPolicyTimeout.describe.skipand// Failing: See #258331comment.The shared page object (
add_cis_integration_form_page.ts) is also used bycis_integration_aws.tsandcis_integration_cnvm.ts— the change is strictly additive (longer timeout, extra loading guard) and does not affect their behaviour.Checklist
Identify risks
Made with Cursor