[CSP] Fix flaky status_indexing API test - add missing Fleet setup call#272018
Draft
animehart wants to merge 3 commits into
Draft
[CSP] Fix flaky status_indexing API test - add missing Fleet setup call#272018animehart wants to merge 3 commits into
animehart wants to merge 3 commits into
Conversation
… wait timeout The 'Add Agent FLyout' tests in the GCP CSPM suite were consistently timing out with: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="agentEnrollmentFlyout"]) Wait timed out after 10039ms Root cause: `retry.waitFor` in `clickFirstElementOnIntegrationTableAddAgent` uses `timeouts.waitFor` from the FTR config (10 s). The agent enrollment flyout can take longer than that to render in CI after clicking Add Agent, especially when the page still has pending network requests. Fix: - Add `waitUntilLoadingHasFinished()` before clicking the Add Agent button so the page is stable before the click. - Switch `retry.waitFor` → `retry.waitForWithTimeout(..., 30_000, ...)` so the flyout has the same 30-second budget used by `saveIntegrationPolicyTimeout`. Also removes the `describe.skip` and the `// Failing: See elastic#258331` comment from group4/pages/cis_integrations/cspm/cis_integration_gcp.ts. Closes elastic#258331 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ssertions The previous implementation used retry.tryForTime and threw when the string was absent, meaning it could never return false. Tests asserting .to.be(false) (e.g. ORG_ID absent in Single Account mode) always timed out with "retry.tryForTime reached timeout 20000 ms". Fix: wait up to 30 s for the code block to be non-empty, then return text.includes(str) as a plain boolean so both .to.be(true) and .to.be(false) assertions work correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
The STATUS = INDEXING TEST suite was consistently returning 'not-deployed' instead of 'indexing' because the beforeEach block was missing the POST /api/fleet/setup call required to initialise Fleet before creating agent and package policies. Without Fleet setup, the CSP status API cannot resolve agent policy references and falls back to 'not-deployed'. The sibling status_indexed.ts suite (non-skipped) already includes this call; this fix brings status_indexing.ts into parity with that pattern. Closes elastic#240000 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
/flaky ftrConfig:x-pack/solutions/security/test/api_integration/apis/cloud_security_posture/config.ts:50 |
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#12535
|
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12535[✅] x-pack/solutions/security/test/api_integration/apis/cloud_security_posture/config.ts: 50/50 tests passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
STATUS = INDEXING TESTsuite instatus_indexing.tswas consistently returningnot-deployedinstead ofindexingbecause thebeforeEachblock was missing thePOST /api/fleet/setupcall required to initialise Fleet before creating agent and package policies.Without Fleet setup, the CSP status API cannot resolve agent policy references and falls back to
not-deployed. The siblingstatus_indexed.tssuite (non-skipped) already includes this call — this fix bringsstatus_indexing.tsinto parity with that pattern.Closes #240000
Checklist
Identify risks
POST /api/fleet/setuptobeforeEachincreases per-test setup time by ~1–2 s in CI. This is acceptable; all other status test suites already pay this cost.