Repository navigation
Failing test: Custom resolution rules - previews, links local users to AD, stops on disable, resumes on enable, and cleans up on delete #299611
Description
A test failed on a tracked branch
Scout Test Details:
| Field | Value |
|---|---|
| Test ID | ddd5b28c53d0781-c77d58aee342dfb |
| Target | cloud-serverless-security_complete |
| Location | x-pack/platform/plugins/shared/entity_store/test/scout/crud_and_resolution/api/tests/custom_resolution_rules.spec.ts |
| Duration | 50.92s |
| Module | entityStore (plugin) |
| Config path | x-pack/platform/plugins/shared/entity_store/test/scout/crud_and_resolution/api/playwright.config.ts |
| Code Owners | elastic/security-entity-analytics |
Error: expect(received).toBeUndefined()
Received: {"deferredByDeadline": false, "definitionFingerprint": "ed44715c0b288b89fae1965e1f1e3337e1a0d78917113d680acbd8cd4f66d93b", "lastAttemptAt": "2026-10-21T15:42:20.558Z", "lastError": null, "lastProcessedTimestamp": "2026-10-21T15:42:03.853Z", "lastRun": {"cascadeRetargeted": 0, "cascadesBlocked": 0, "resolutionsCreated": 1, "skippedAmbiguousBuckets": 0, "skippedNoopBuckets": 0, "skippedOversizedBuckets": 0}, "lastStartedFirstAt": "2026-10-21T15:42:20.558Z"}
at x-pack/platform/plugins/shared/entity_store/test/scout/crud_and_resolution/api/tests/custom_resolution_rules.spec.ts:402:76
at x-pack/platform/plugins/shared/entity_store/test/scout/crud_and_resolution/api/tests/custom_resolution_rules.spec.ts:388:9
First failure: appex-qa-serverless-kibana-scout-tests - main
Test history: passes and failures over time
Activity
github-actions commentedon Oct 21, 2026
Test needs an update — one maintainer run isn't a guaranteed prune
custom_resolution_rules.spec.ts:363asserted the deleted rule's saved progress was gone after a singlesync=truemaintainer run, but read back — unchanged — the state the preceding "re-enabled" run wrote (lastAttemptAt=lastStartedFirstAt=15:42:20.558Z,lastRun.resolutionsCreated: 1), i.e. the prune never landed (medium confidence: Cloud serverless ships no Kibana logs, so the exact path can't be pinned).Details
Proposed fix
In the
delete removes the rule and its saved progress, links staystep, replace the one-shot assertion atcustom_resolution_rules.spec.ts:363with a bounded convergence wait: pollreadCustomRuleState(apiClient, ruleId)until it isundefined, re-issuingtriggerMaintainerRun(..., { sync: true })between attempts, and fail with the last-read state. Pruning only happens inside a run, so a read-only poll can't converge within the 5-minute task interval — re-triggering is required, and it is safe here because the maintainer is an idempotent reconciliation task the product itself re-runs every 5 minutes.helpers.ts:650-655already does exactly this (trigger → read → trigger → read) for the SID watermark; generalise it into a sharedwaitForMaintainerState-style helper rather than open-coding a second copy. This keeps the test's intent intact — the progress is pruned — while matching the guarantee the product offers, per Design tests with a cloud-first mindset.Subordinate alternative, if the team wants one run to always prune: make
pruneDeletedRulesderive its candidate set from thecustom:state keys alone (confirming each with the realtime by-id read) and run it before the rule listing, so pruning no longer depends on the eventually-consistent listing or on the listing succeeding at all.Root cause & evidence
The state read back is byte-for-byte what the previous step's run wrote, which is what "the prune never ran" looks like — not "the prune ran and was wrong".
runCustomResolutionRulesis deliberately best-effort about that prune, and at least three paths through it leave the entry in place; all three are consistent with the observed state, and nothing in the available artifacts separates them:pruneDeletedRuleswhenever the rule listing rejects (custom_rules.ts:151-158) — a transient saved-objects/PIT error, whichpipelines.mdcalls out as a Cloud characteristic, is enough.findCustomResolutionRules,client.ts:101-117) while the confirmation read is a realtimesoClient.get(client.ts:119-130). The test's 404 at line 360 goes through the realtime path, so it can pass while the search still lists the rule as live and the key is never even considered for pruning.syncroute against the maintainer's own 5-minute Task Manager tick:runSyncreads the task state, runs, then writes it back wholesale, so a tick already in flight across the delete finishes afterwards and writes back its pre-delete snapshot.The test is the only place that assumes one run settles this; the same fixture file's own helper already assumes it doesn't (
helpers.ts:650-655triggers, reads, then triggers and reads again). The spec is two days old — added by #297506 (merged 2026-10-19), which introduced both the assertion and the best-effort prune — and this is its first failure, oncloud-serverless-security_completeonly.Additional context
createPlaywrightConfigdefaults toworkers: 1and Cloud provisions a project per config, so no sibling spec could have been triggering runs); a stale read in the assertion itself (getMaintainersreads the task document with a realtimetaskManager.get, not a search).test-failures-summary.json— Cloud serverless runs ship no.scout/server.log— so I could not pin which of the three paths fired.entityStoremaintainer logs atinfo/warnfor the window 15:42:20–15:42:35Z would settle it in one line:Skipping custom resolution rules this run; failed to load them: ...(listing failed),custom:<id>: keeping its state; ...(by-id read threw), or two overlappingCustom resolution rules this run, in order: ...lines (a concurrent tick). Capturingplugins.entityStoreatinfoand shipping the Kibana log with Cloud serverless Scout artifacts would make the next occurrence conclusive.node scripts/scout.js run-tests --location cloud --arch serverless --domain security_complete --config x-pack/platform/plugins/shared/entity_store/test/scout/crud_and_resolution/api/playwright.config.tsTip
Marked "AI-fixable". Add
ai:fix-flakyto request a fix now; otherwise it will be requested automatically if the test fails again.infra-vault-gh-plugin-prod commentedon Oct 21, 2026
Pinging @elastic/security-entity-analytics (Team:Entity Analytics)
kibanamachine commentedon Oct 24, 2026
New failure for "cloud-serverless-security_complete" target: appex-qa-serverless-kibana-scout-tests - main
Test history: passes and failures over time
New error message:
github-actions commentedon Oct 24, 2026
✅ Flaky Test Fixer completed successfully!
github-actions commentedon Oct 24, 2026
➡️ A fix PR is ready for review: #300346
The maintainer now prunes a deleted custom rule's saved progress from the realtime read-by-id rather than from the eventually consistent rule listing, so a single
sync=truerun prunes it as the spec expects.