Migrate Actionable Observability-owned FTR suites to Scout

[shahzad31]

Summary

Track migration of Functional Test Runner (FTR) suites that are owned by @elastic/actionable-obs-team (per .github/CODEOWNERS) to Scout (Playwright-based), so we reduce FTR maintenance cost and align with the platform direction for new functional/API tests.

This issue is meant as a parent epic; consider splitting work into smaller issues or PRs per area (e.g. uptime/synthetics vs observability API integration vs alerting).

---

Progress

Merged

• [Observability] Replace basic-license annotations FTR test with unit coverage !! #270174 — [Observability] Replace basic-license annotations FTR test with unit coverage — removes observability_api_integration/basic config (merged 2026-05-21)
• [Uptime] Merge scout_uptime_legacy into scout test suite #270146 — [Uptime] Consolidate scout_uptime_legacy into the main test/scout/ suite (cleanup follow-up to the uptime migrations) (merged 2026-05-22)
• [Uptime] Migrate FTR API tests to Scout #269978 — [Uptime] Migrate FTR API tests to Scout — removes api_integration/apis/uptime FTR config (merged 2026-05-21)
• [SLO] Migrate slo/embeddables FTR test to Scout !! #270121 — [SLO] Migrate slo/embeddables FTR test to Scout (merged 2026-05-20)
• Migrate uptime FTR functional tests to Scout !! #269985 — Migrate uptime FTR functional tests to Scout (merged 2026-05-20)
• [scout] start migrating Synthetics deployment-agnostic API tests #263963 — [scout] Start migrating Synthetics deployment-agnostic API tests — Phase 1 scaffold + 2 specs (merged 2026-05-18)
• [SLO] Migrate synthetics to scout !! #225035 — [SLO] Migrate synthetics-based SLO plugin tests to Scout (earlier groundwork, merged 2025-07-14)

In progress (open draft PRs)

• [Observability] Migrate alerting_api_integration FTR tests to Scout API #272035 — [Observability] Migrate alerting_api_integration/observability (metric threshold rule + custom threshold rule data view) to a new Scout API suite — removes alerting_api_integration/ (draft, opened 2026-05-30)
• [Synthetics] Migrate obs security api_keys FTR test to Scout API #272034 — [Synthetics] Migrate api_integration/apis/security/api_keys (API key creation) into the Synthetics Scout API suite — removes api_integration/apis/security/config.ts (draft, opened 2026-05-30)
• [Uptime] Migrate functional_with_es_ssl alert tests to Scout #272030 — [Uptime] Migrate functional_with_es_ssl alert tests to Scout (uptime monitor-status alert flyout + simple down alert UI) — removes functional_with_es_ssl/config.ts (draft, opened 2026-05-30)
• [Observability] Migrate observability API integration tests to Scout #272029 — [Observability] Migrate observability_api_integration (trial) to Scout API tests — annotations CRUD + SLO annotation filtering + AI assistant alert context; removes observability_api_integration/ (draft, opened 2026-05-30)
• [Synthetics] Migrate monitor create/edit API tests to Scout (batch 1) #272008 — [Synthetics] Migrate monitor create/edit deployment-agnostic API tests to Scout (batch 1) (draft, opened 2026-05-29)
• [Synthetics] Migrate monitor delete/edit/get/test API tests to Scout (batch 2) #272010 — [Synthetics] Migrate monitor delete/edit/get/test-now deployment-agnostic API tests to Scout (batch 2, stacked on [Synthetics] Migrate monitor create/edit API tests to Scout (batch 1) #272008) (draft, opened 2026-05-29)
• [synthetics/ftr] cache package install + share private location #263966 — [synthetics/ftr] Cache package install + share private location (FTR-side optimizations split out from [scout] start migrating Synthetics deployment-agnostic API tests #263963; helps de-risk the remaining synthetics migration)

---

Scope: FTR configs to migrate

These paths are listed in observability FTR manifests (.buildkite/ftr_oblt_stateful_configs.yml / .buildkite/ftr_oblt_serverless_configs.yml) and fall under CODEOWNERS rules for @elastic/actionable-obs-team. Test counts are approximate (it(...) occurrences in test files at the time of writing).

Stateful (ftr_oblt_stateful_configs.yml)

• x-pack/solutions/observability/test/observability_api_integration/basic/config.ts — ~1 test — done in [Observability] Replace basic-license annotations FTR test with unit coverage !! #270174 (replaced with unit coverage)
• x-pack/solutions/observability/test/observability_api_integration/trial/config.ts — ~29 tests (3 spec files) — in progress in [Observability] Migrate observability API integration tests to Scout #272029
• x-pack/solutions/observability/test/observability_functional/with_rac_write.config.ts — ~92 tests (16 spec files; alerts/cases/overview/rules pages + feature controls)
• x-pack/solutions/observability/test/functional_with_es_ssl/config.ts — ~26 tests (2 spec files: uptime alert flyout + simple down alert) — in progress in [Uptime] Migrate functional_with_es_ssl alert tests to Scout #272030
• x-pack/solutions/observability/test/functional/apps/slo/embeddables/config.ts — ~10 tests — done in [SLO] Migrate slo/embeddables FTR test to Scout !! #270121
• x-pack/solutions/observability/test/alerting_api_integration/observability/config.ts — ~13 tests (2 spec files) — in progress in [Observability] Migrate alerting_api_integration FTR tests to Scout API #272035
• x-pack/solutions/observability/test/api_integration/apis/security/config.ts — ~1 test — in progress in [Synthetics] Migrate obs security api_keys FTR test to Scout API #272034
• x-pack/solutions/observability/test/api_integration/apis/synthetics/config.ts — ~81 tests (13 spec files)
• x-pack/solutions/observability/test/api_integration/apis/uptime/config.ts — ~38 tests (13 spec files) — done in [Uptime] Migrate FTR API tests to Scout #269978
• x-pack/solutions/observability/test/functional/apps/uptime/config.ts — ~38 tests (8 spec files) — done in Migrate uptime FTR functional tests to Scout !! #269985

Serverless (ftr_oblt_serverless_configs.yml)

• x-pack/solutions/observability/test/api_integration_deployment_agnostic/configs/serverless/oblt.synthetics.serverless.config.ts — ~274 tests remaining across ~29 spec files; scaffold + first 2 specs already migrated in [scout] start migrating Synthetics deployment-agnostic API tests #263963; FTR-side prep in [synthetics/ftr] cache package install + share private location #263966; deployment-agnostic monitor CRUD batches in progress in [Synthetics] Migrate monitor create/edit API tests to Scout (batch 1) #272008 (batch 1) and [Synthetics] Migrate monitor delete/edit/get/test API tests to Scout (batch 2) #272010 (batch 2)

Related (CODEOWNERS: actionable obs; not a standalone FTR entry in the manifests)

• src/platform/test/api_integration/apis/suggestions — ~12 tests; owned by actionable obs; confirm whether tests are only pulled in via broader API integration configs and migrate coverage accordingly.

---

Guidance for Scout migration

1. Decide the right layer (avoid “FTR parity” for its own sake)

• Prefer unit / RTL for isolated UI logic; Scout API tests for data correctness; Scout UI tests for real user journeys and rendering/interaction.
• If a test mostly asserts API payloads or document counts, migrate it to a Scout API spec (or keep/ add server-side tests), not a browser spec.
• See: docs/extend/scout — start with Getting started, Best practices, Write API tests, Write UI tests.

2. Scaffold and location

• Add Scout tests under the owning plugin/package test/scout/ tree (see existing examples in observability plugins, e.g. observability, slo, infra).
• Follow Setup Plugin for scout config in kibana.jsonc and Playwright wiring.

3. FTR → Scout mapping

• Replace loadTestFile()-style suites with standalone specs; do not rely on file execution order.
• Map FTR services / page objects to Scout fixtures and page objects (docs/extend/scout/fixtures.md, page-objects.md).
• Ensure setup is space-safe and parallel-safe where tests run in parallel (parallelism, deployment tags).

4. Running tests locally

From repo root (see Run tests):

node scripts/scout.js run-tests --stateful --config <path-to-scout-config>
# or --testFiles for specific specs

5. AI-assisted migration (optional)

• Experimental prompts and notes for FTR→Scout migration live under src/platform/packages/private/kbn-scout-info/llms/ — treat output as review-required, not copy-paste.

6. CI / ownership

• After migration, remove or narrow FTR config usage and update CI/manifests as required by the Appex QA / build pipeline expectations for Scout.
• Coordinate with @elastic/observability-ui and other teams where CODEOWNERS overlap (many observability FTR paths are shared across teams).

---

Definition of done (per area)

• Equivalent coverage exists under Scout (or explicitly justified as moved to API/unit tests).
• FTR config(s) for that area removed or disabled with a clear follow-up if anything is deferred.
• Documentation or README updated if the plugin has a test/scout/README.md pattern.

---

References

• CODEOWNERS — Actionable Observability block (search for actionable-obs-team and “Actionable Observability”).
• AGENTS.md (repo root): Scout command for running tests.