Commit 80cdc4c
[Agent Builder] Skip esql specs in agent-builder evals run (elastic#267522)
## Summary
- Adds `testIgnore: ['**/esql/**']` to the agent-builder Playwright eval
config so the recursive `testDir` walk no longer picks up
`evals/esql/esql.spec.ts`.
- That spec is owned by the separate `esql-generation` suite
(`esql.playwright.config.ts`), so today it runs twice every weekly LLM
evals cycle — ~4 min/run × ~7 connectors = **~28 min/week of duplicate
work** in the agent-builder child.
- Same recursive-`testDir` bug pattern fixed for entity-analytics v1 in
elastic#267451.
Refs elastic/security-team#17139
## Stacking
Stacked on top of elastic#267451 (`evals/ea-v1-ignore-v2-specs`), which extends
`createPlaywrightEvalsConfig` to forward the optional `testIgnore`. Will
retarget to `main` once elastic#267451 merges.
## Test plan
- [ ] `node scripts/type_check --project
x-pack/platform/packages/shared/agent-builder/kbn-evals-suite-agent-builder/tsconfig.json`
clean
- [ ] `node scripts/eslint
x-pack/platform/packages/shared/agent-builder/kbn-evals-suite-agent-builder/playwright.config.ts`
clean
- [ ] Next weekly `kibana-evals-weekly-llm-evals` run: agent-builder
Playwright report no longer lists `evals/esql/esql.spec.ts`, p50 drops
by ~4 min vs `build_13` baseline
- [ ] `esql-generation` suite continues to pass on its own config
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>1 parent 587b97c commit 80cdc4c
2 files changed
Lines changed: 8 additions & 0 deletions
File tree
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
12 | 17 | | |
13 | 18 | | |
14 | 19 | | |
| |||
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| 33 | + | |
32 | 34 | | |
33 | 35 | | |
34 | 36 | | |
| |||
106 | 108 | | |
107 | 109 | | |
108 | 110 | | |
| 111 | + | |
109 | 112 | | |
110 | 113 | | |
0 commit comments