[Entity Analytics][Lead Generation] Foundation: feature flag, shared types, index templates, observation module interface by abhishekbhatia1710 · Pull Request #2 · abhishekbhatia1710/kibana

abhishekbhatia1710 · 2026-02-27T09:43:38Z

Summary

Adds the foundational layer for the Entity Analytics Lead Generation feature:

Feature flag (leadGenerationEnabled) to gate the entire pipeline
Shared types and Zod schemas for leads, observations, entities, engine config, and API contracts
Elasticsearch index templates with mappings and lifecycle service for lead persistence (adhoc + scheduled modes)
Observation module interface and registry for pluggable, priority-ordered, fault-tolerant module execution

This PR contains no runtime wiring — it establishes the contracts and infrastructure that subsequent PRs will build on.

Relates to: https://github.com/elastic/security-team/issues/15946

Testing

Unit tests — run the following commands individually:

yarn test:jest x-pack/solutions/security/plugins/security_solution/common/entity_analytics/lead_generation/types.test.ts
yarn test:jest x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/indices/lead_index_service.test.ts
yarn test:jest x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/observation_module_registry.test.ts

Type check — verify no type errors:

yarn test:type_check --project x-pack/solutions/security/plugins/security_solution/tsconfig.json

Feature flag — confirm leadGenerationEnabled defaults to false and does not alter existing behavior when disabled.

…red types, index templates, observation module interface

…#258775) ## Summary Creates the foundation package `@kbn/evals-extensions` for advanced evaluation capabilities. This package will house features ported from cursor-plugin-evals and serve as the home for Phases 3-5 of the evals roadmap. ## Architecture **One-way dependency:** - ✅ kbn-evals-extensions depends on kbn-evals - ❌ kbn-evals has NO dependency on kbn-evals-extensions Evaluation suites opt-in by importing from extensions directly. ## What's Included ✅ Package structure and build configuration ✅ Comprehensive documentation ✅ 5 passing unit tests ✅ CODEOWNERS entry ✅ No functional changes ## Validation ✅ Bootstrap, type check, tests, eslint, check_changes.ts all passed ✅ No circular dependencies ## Roadmap This enables PRs #2-10 for cost tracking, dataset management, safety evaluators, UI components, DX enhancements, analytics, A/B testing, human-in-the-loop, and IDE integration. ## Related - Part of elastic#257821 - Enables elastic#257823, elastic#257824, elastic#257825, elastic#257826 - Addresses elastic#255820 Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Garrett Spong <garrett.spong@elastic.co>

@timestamp

Closes elastic#258318 Closes elastic#258319 ## Summary Adds logic to the alert episodes table to display `.alert_actions` information. This includes: - New action-specific API paths. - Snooze - **Per group hash.** - Button in the actions column opens a popover where an `until` can be picked. - **When snoozed** - A bell shows up in the status column. - Mouse over the bell icon to see until when the snooze is in effect. - Unsnooze - **Per group hash.** - Clicking the button removes the snooze. - Ack/Unack - **Per episode.** - Button in the actions column - When "acked", an icon shows in the status column. - Tags - This PR only handles displaying tags. They need to be created via API. - Resolve/Unresolve - **Per group hash.** - Button inside the ellipsis always - The status is turned to `inactive` **regardless of the "real" status.** <img width="1704" height="672" alt="Screenshot 2026-03-25 at 16 04 12" src="https://github.com/user-attachments/assets/5ef4111a-6e0c-4114-a60e-ce5f81a86ac6" /> ## Testing <details> <summary>POST mock episodes</summary> ``` POST _bulk { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:00:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:01:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:02:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:03:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:04:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:05:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:06:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-001", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:07:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:08:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:09:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:10:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:11:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:12:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:13:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-2", "episode": { "id": "ep-002", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "gh-1", "episode": { "id": "ep-003", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-4", "episode": { "id": "ep-004", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-4", "episode": { "id": "ep-004", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-4", "episode": { "id": "ep-004", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:19:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-4", "episode": { "id": "ep-004", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-5", "episode": { "id": "ep-005", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:22:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-5", "episode": { "id": "ep-005", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-006", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-006", "status": "active" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:26:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-1" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-006", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:14:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "elasticgh-7", "episode": { "id": "ep-007", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:15:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-2" }, "group_hash": "elasticgh-7", "episode": { "id": "ep-007", "status": "inactive" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:16:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "elasticgh-8", "episode": { "id": "ep-008", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:17:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "elasticgh-8", "episode": { "id": "ep-008", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:18:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-3" }, "group_hash": "elasticgh-8", "episode": { "id": "ep-008", "status": "recovering" }, "status": "recovered" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:20:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:21:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-4" }, "group_hash": "elasticgh-9", "episode": { "id": "ep-009", "status": "pending" }, "status": "no_data" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:23:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "pending" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:24:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "breached" } { "create": { "_index": ".rule-events" }} { "@timestamp": "2026-01-27T16:25:00.000Z", "source": "internal", "type": "alert", "rule": { "id": "rule-5" }, "group_hash": "elasticgh-10", "episode": { "id": "ep-010", "status": "active" }, "status": "no_data" } ``` </details> - In the POST above, episodes 1 and 3, and episodes 6 and 9 have the same group hashes. - Go to `https://localhost:5601/app/observability/alerts-v2` and try all buttons. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…260054) ## Summary issue: elastic/security-team#15982. (Resolves requirement #2) Enriches `SelectionContext` with step `values` so that `PropertySelectionHandler` implementations (`search`, `resolve`, `getDetails`) can access sibling property values from the current step definition. There are no existing steps using `context.values` yet — this is a prerequisite for upcoming steps from the Cases team that need to read sibling properties (e.g. `owner`) to scope their search/resolve logic. ### What changed **`SelectionContext.values`** — A new `values` field (`{ config, input }`) is populated from the step's YAML properties at the time `search`/`resolve`/`getDetails` are called. Handlers can now read sibling values like `context.values.input.owner` instead of having no visibility into the rest of the step. **YAML value extraction fix** — `getValueFromValueNode` now handles non-scalar YAML nodes (arrays/sequences) via `.toJSON()`, fixing a bug where array properties like `owner: [securitySolution]` appeared as `undefined` in `context.values`. **Example** — An example has been implemented in the `examples.externalStep` from the _workflows_examples_ plugin (test with `node scripts/kibana --dev --run-examples`) ### Demo https://github.com/user-attachments/assets/b0cf18ae-5906-4561-bf5e-31b228b08a30 ### Files changed | Area | Files | Change | |------|-------|--------| | Shared types | `kbn-workflows/types/v1.ts`, `latest.ts` | Added `StepSelectionValues` interface, `values` field on `SelectionContext`, generic type parameters on 4 interfaces | | Step registry | `workflows_extensions/.../step_registry/types.ts` | Thread `Config`/`Input` schema types into `StepPropertyHandler` for automatic inference | | Value builder | `build_workflow_lookup.ts` | Added `buildStepSelectionValues()` helper + fixed `getValueFromValueNode` for non-scalar nodes | | Context wiring | `collect_all_custom_property_items.ts`, `get_custom_property_suggestions.ts` | Pass `values` when constructing `SelectionContext` | | Docs | `STEPS.md` | Updated `SelectionContext` type definition and documented `context.values` | | Tests | `build_workflow_lookup.test.ts`, `validate_custom_properties.test.ts`, `get_custom_property_suggestions.test.ts` | New tests for value building (including arrays), updated assertions for new context shape | --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

elastic#270540) ## Release note When using Kibana Spaces, the Synthetics monitor health endpoint could incorrectly report monitors as unhealthy — showing errors such as "missing location", "missing agent policy", or "missing package policy" — even when everything was properly configured. This happened because the health check was only looking for monitors, private locations, and Fleet policies in the current space, missing resources that existed in other spaces. These issues are now fixed: the health check correctly resolves monitors, private locations, package policies, and agent policies across all relevant spaces, giving an accurate health status regardless of how resources are distributed across your Kibana Spaces. ## Summary Closes elastic#270477. `POST /internal/synthetics/monitors/_health` returned wrong results when monitors lived outside the request's space — `missing_package_policy` errors when called from the monitor's space, and 404s when called from `default`. Two independent space-scoping bugs: 1. **Package policy lookup ignored space.** `getExistingPackagePoliciesMap` called Fleet's `packagePolicyService.getByIDs` with `createInternalRepository()`, which is scoped to the default namespace. Package policies created for monitors in another space were therefore invisible. 2. **Monitor saved-object lookup was space-scoped.** `MonitorConfigRepository.get` used the request-scoped saved-objects client, restricted to the request's space. Calling `_health` from `default` for a monitor that lives elsewhere returned a 404. ## What changed - **`PackagePolicyService.getByIds`** — accepts a new optional `additionalSpaceIds`, so the wrapper's per-space scoped-client fan-out can broaden beyond `[spaceId, default]`. Existing callers keep their old behavior. - **`MonitorConfigRepository.getAcrossSpaces(id, namespaces, soClient?)`** — new method that resolves a monitor across an arbitrary list of spaces. Uses the multi-space type's per-object `namespaces` array in one bulkGet entry, plus one entry per namespace for the `namespaceType: 'single'` legacy type. Accepts an injected `soClient` so the health API can pass `createInternalRepository()` and bypass the request's space restriction. - **`MonitorIntegrationHealthApi`**: - Computes `allSpaces = { requestSpace, ...allSpacesWithMonitors }` once, up-front. - `fetchMonitors` calls `monitorConfigRepository.getAcrossSpaces` with the internal repository → fixes bug #2. - `getExistingPackagePoliciesMap` uses the `PackagePolicyService` wrapper with `additionalSpaceIds` → fixes bug #1. ## Test plan - [x] `node scripts/jest` on the three affected suites — **77/77 passing** (includes new cross-space coverage and a new `getAcrossSpaces` test block). - [x] `node scripts/type_check --project x-pack/solutions/observability/plugins/synthetics/tsconfig.json` — clean. - [x] `node scripts/eslint` on the changed files — clean. - [ ] Manual: create a monitor with a private location in a non-default space, then call `POST /internal/synthetics/monitors/_health` both from that space and from `default`. Verify each call reports the monitor accurately instead of `missing_package_policy` / 404. ## Related - elastic#270137 — related health API fix (project monitor policy ID + infinite polling). Made with [Cursor](https://cursor.com) --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Miguel Martín <miguel.martin@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Fixes elastic#261258 possibly also elastic#264565 (to be verified) ## Summary Add `meta: { id }` to Fleet body/response schemas Files changed: - `server/types/models/agent_policy.ts` — versioned policy schemas (v3–v6), `new_agent_policy`, `agent_policy`, `agent_policy_response`, `full_agent_policy`, outputs responses - `server/types/rest_spec/agent_policy.ts` — bulk get, copy, delete, full policy, K8s manifest, list outputs, cleanup task request/response schemas - `server/types/models/enrollment_api_key.ts `— `enrollment_api_key` - `server/types/rest_spec/enrollment_api_key.ts` — enrollment key CRUD and bulk delete schemas - `common/types/models/package_policy_schema.ts `— all package policy schemas: new, versioned (v22–v24), simplified, update, response, dry-run, status - `server/types/rest_spec/package_policy.ts` — bulk get, create, delete, upgrade, dry-run schemas - `server/types/rest_spec/epm.ts `— ~40 package management schemas (categories, package info/list/stats, install/delete/bulk operations, etc.) - `server/types/rest_spec/agent.ts` — `action_id_response` / `action_message_response` branches of `ActionIdOrMessageSchema` (resolves items #1 and #2 of elastic#264565) ### Testing Verified locally with: ``` node scripts/capture_oas_snapshot --no-serverless --include-path /api/fleet node scripts/validate_oas_docs node scripts/check_api_contracts --distribution stack ``` It looks like there are no Fleet errors left after this change but the final verification step will be ran in the ci for Terraform --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

[Entity Analytics][Lead Generation] Add foundation: feature flag, sha…

0e43f8e

…red types, index templates, observation module interface

abhishekbhatia1710 closed this Feb 27, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Entity Analytics][Lead Generation] Foundation: feature flag, shared types, index templates, observation module interface#2

[Entity Analytics][Lead Generation] Foundation: feature flag, shared types, index templates, observation module interface#2
abhishekbhatia1710 wants to merge 1 commit into
mainfrom
ea-15946-lead-gen-foundation

abhishekbhatia1710 commented Feb 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

abhishekbhatia1710 commented Feb 27, 2026

Summary

Testing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant