Skip to content

[Response Ops] Migrate rules_list and rule_details FTR specs to Scout#272025

Draft
guskovaue wants to merge 10 commits into
elastic:mainfrom
guskovaue:response-ops/scout-hard-tier-migrations
Draft

[Response Ops] Migrate rules_list and rule_details FTR specs to Scout#272025
guskovaue wants to merge 10 commits into
elastic:mainfrom
guskovaue:response-ops/scout-hard-tier-migrations

Conversation

@guskovaue
Copy link
Copy Markdown
Contributor

@guskovaue guskovaue commented May 30, 2026

Summary

Migrates two Hard-tier FTR specs from the Response Ops FTR → Scout backlog (#266609) to Playwright/Scout.

rules/rules_list/rules_list.ts → rules_list.spec.ts

  • 21 of 26 tests migrated
  • 5 tests skipped (test.skip) — require test.failing or test.always-firing rule types not available in Scout's stateful/classic environment
  • test.noop / test.always-firing → .es-query; test.failing → test.skip
  • FTR preconfigured Slack#xyztest connector replaced by a dynamically-created .slack connector
  • notify_when → notifyWhen, throttle: null → throttle: undefined for Scout's typed API helper

rules/details.ts → 3 separate files (required by scout_max_one_describe)

  • rule_details_header.spec.ts — 8 tests: renders rule details, interval-warning toast, disable/snooze/re-enable, snooze-schedule CRUD
  • rule_details_edit.spec.ts — 2 tests: open edit flyout & save; cancel edit with unsaved-changes confirm modal
  • rule_details_so_navigation.spec.ts — 2 tests: navigate to rule details via Saved Objects Management in default and non-default spaces
  • 6 tests skipped — require preconfigured Slack#xyztest / my-server-log connectors or test.always-firing for alert-instance generation
  • test.always-firing → .es-query with 1s interval (triggers interval-warning toast); .index-threshold kept for edit-flow tests that need action groups
  • FTR preconfigured my-server-log replaced by dynamically-created .server-log connector

Migration patterns applied

  • tags.stateful.classic for all specs
  • browserAuth.loginAsAdmin() in beforeEach
  • apiServices.alerting.rules.create/delete + kbnClient.request for setup/teardown
  • pageObjects.ruleDetailsPage.gotoById(ruleId) for rule-details navigation
  • Status dropdown: statusDropdownDisabledItem / statusDropdownEnabledItem test-subjs (replaces FTR's fragile array-index selection)
  • Cleanup via Promise.allSettled in afterAll
  • Max describe depth = 1 (one root-level describe per file)

Skipped tests tracking

The following tests are skipped pending upstream work:

Test Reason
should filter alerts by the status Needs test.failing rule type
should display total alerts by status and error banner Needs test.failing rule type
Expand error in rules table when there is a rule with an error Needs test.failing rule type
should untrack disable rule if untrack switch is true Needs test.always-firing (specific alert lifecycle)
should not untrack disable rule if untrack switch is false Needs test.always-firing (specific alert lifecycle)
should show and update deleted connectors Needs preconfigured Slack#xyztest connector
renders the active alerts Needs test.always-firing for named alert instances
renders the muted inactive alerts Needs test.always-firing for named alert instances
renders the first page Needs test.always-firing for alert pagination
navigates to the next page Needs test.always-firing for alert pagination
renders the event log list Complex event-log data-grid verification; deferred

Test plan

  • node scripts/eslint.js — no errors on all 4 new spec files ✅
  • node scripts/scout.js update-test-config-manifests --includingUpToDate — manifests updated ✅
  • CI stateful/classic run against the 4 new spec files

🤖 Generated with Claude Code

@guskovaue @claude
[Response Ops] Migrate rules_list and rule_details FTR specs to Scout
8f4d7ab 
Migrates two "Hard" tier specs from the FTR → Scout backlog
(x-pack/platform/test/functional_with_es_ssl/apps/rules/):

- rules/rules_list/rules_list.ts (26 tests) → rules_list.spec.ts
  21 tests migrated; 5 skipped (need test.failing/test.always-firing)

- rules/details.ts (20 tests) → 3 separate files (scout_max_one_describe):
  • rule_details_header.spec.ts (8 tests – disable/snooze/re-enable flow)
  • rule_details_edit.spec.ts (2 tests – edit flyout / cancel with confirm)
  • rule_details_so_navigation.spec.ts (2 tests – SO management nav)
  6 tests skipped (need preconfigured connectors or always-firing rule type)

Rule type substitutions: test.noop / test.always-firing → .es-query;
test.failing → test.skip. FTR preconfigured connectors replaced with
dynamically-created .server-log / .slack connectors where possible.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@infra-vault-gh-plugin-prod
Copy link
Copy Markdown

infra-vault-gh-plugin-prod Bot commented May 30, 2026

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!
  • Click to trigger kibana-entity-store-performance-from-pr for this PR!
  • Click to trigger kibana-storybooks-from-pr for this PR!

guskovaue and others added 9 commits May 30, 2026 20:46
@guskovaue @claude
[Response Ops] Migrate Hard-tier connector and alert flyout FTR specs…
aa24e1b 
… to Scout

Add Scout/Playwright specs for 5 Hard-tier files from the migration backlog:
- connector_email.spec.ts (6 tests migrated from email.ts)
- connector_general.spec.ts (10 migrated + 3 skipped from general.ts)
- connector_jsm.spec.ts (19 tests migrated from jsm.ts)
- connector_opsgenie.spec.ts (19 tests migrated from opsgenie.ts)
- alert_create_flyout.spec.ts (5 migrated + 8 skipped from alert_create_flyout.ts)

Skipped tests require preconfigured connectors (Slack#xyztest, test-preconfigured-email,
Serverlog), rule types not available in Scout stateful/classic (test.always-firing,
apm.error_rate), or unavailable services (filterBar.addDslFilter).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops] Unskip preconfigured-connector delete test in Scout
e457397 
Add a 'my-server-log' (.server-log) preconfigured connector to the default
Scout stateful classic server config, mirroring what the FTR base config does.
This unblocks the 'should not be able to delete a preconfigured connector' test
in connector_general.spec.ts which was skipped for lack of a preconfigured
connector in the Scout environment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops] Unskip preconfigured-connector edit test in Scout
1a55faf 
Add 'my-email-connector' (.email) to the default Scout stateful classic
preconfigured connectors, alongside the 'my-server-log' added previously.
This unblocks the 'should not be able to edit a preconfigured connector' test
in connector_general.spec.ts: verifies that opening a preconfigured connector
shows the preconfigured badge and hides the save button.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops] Unskip Execution log test in connector_general Scout spec
502db34 
Implements the previously-skipped execution log test using an .es-query
rule (threshold >= 0) as a substitute for the unavailable test.always-firing
rule type. Uses _run_soon + polling to ensure at least one execution is
recorded before asserting on the event log UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops] Unskip 6 alert flyout tests in Scout spec
7e274db 
Implements tests 1-6 from alert_create_flyout.spec.ts that were
previously skipped due to missing Slack#xyztest preconfigured connector
and unavailable test.always-firing rule type.

Substitutions:
- Slack#xyztest → Slack connector created via API in beforeAll
- defineIndexThresholdAlert → Scout Playwright helper (tests 1-4)
- test.always-firing → .es-query rule type (tests 5-6)

Tests 11 (APM rule type) and 13 (filterBar.addDslFilter) remain
skipped as no Scout equivalent exists for those dependencies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops][Scout] Unskip all remaining tests in alert_create_flyo…
486bf0e 
…ut.spec.ts

Implements the final 2 skipped tests to complete 13/13 migration:
- 'should successfully show the APM error count rule flyout': navigates to
  create rule dialog, selects apm.error_rate, asserts APM-specific fields visible
- 'should add filter': navigates to triggersActionsAlerts page and adds a DSL
  filter via the filter bar, verifying the filter badge appears

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Scout] Update .meta standard.json and parallel.json manifests
af4722d 
Commit accumulated Scout test metadata updates across multiple plugins:
new entries, renamed file paths, and updated SHA1 checksums generated
during the Scout migration session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude
[Response Ops][Scout] Unskip all 5 remaining tests in rules_list.spec.ts
5074377 
Implements the final 5 skipped tests to complete 26/26 migration:

- 'should filter alerts by the status': creates a normal + failing .es-query
  rule (invalid query type → ES rejects at execution time), runs both, then
  polls via ruleLastRunOutcomeFilterButton until the failed rule appears.

- 'should display total alerts by status and error banner': verifies
  rulesErrorBanner appears and totalSucceeded/Failed/Warning counts are correct
  after a failing rule executes (search text scopes the aggregation to our
  tagged rules so counts are exact).

- 'Expand error in rules table when there is rule with an error associated':
  verifies expandRulesError link appears only after a failing rule executes,
  and the expanded row contains 'Error from last run'.

- 'should untrack disable rule if untrack switch is true': creates an always-
  firing .es-query rule (threshold >= 0), waits for alert instance 'query
  matched' to appear in _alert_summary, disables with untrackAlertsModalSwitch
  toggled on, and asserts tracked becomes false.

- 'should not untrack disable rule if untrack switch if false': same setup,
  disables without toggling the switch, asserts tracked remains true.

New helpers: FAILING_ES_QUERY, runRuleSoon, getAlertSummary,
createFailingRule, createAlwaysFiringRule.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guskovaue @claude @cursoragent
[Response Ops][Scout] Restore missing rule-details parity test and re…
ea83b48 
…fresh manifest

Adds the FTR "Edit rule with legacy rule-level notify values" test
(rules/details.ts) that was dropped during migration — it was neither
migrated nor listed as skipped. It verifies that a rule created with
legacy rule-level notify_when/throttle surfaces those values as
action-level frequency in the edit flyout and saves successfully.
Uses .index-threshold + a dynamically created .server-log connector.

Also regenerates test/scout/.meta/ui/standard.json: the committed
manifest was stale and still recorded the previously-unskipped
alert_create_flyout, connector_general and rules_list tests as
"skipped", which would mislead CI test planning.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@guskovaue