[Vis skill] Fixes ES|QL escaping drift during visualization config generation#272493
Conversation
|
Pinging @elastic/kibana-presentation (Team:Presentation) |
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
|
|
Starting backport for target branches: 9.4 |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
3 similar comments
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
…neration (elastic#272493) ## Summary Fixes ES|QL escaping drift during visualization config generation. ## What Was Not Working `generate_esql` produced the correct query for a GROK pattern: ```esql | GROK message "%{IP} - - \\[%{DATA}\\] \"%{WORD:method} %{DATA}\"" ``` But when the visualization config was generated, the query embedded in `visualization.data_source.query` lost one escaping level for the bracket regex: ```esql | GROK message "%{IP} - - \[%{DATA}\] \"%{WORD:method} %{DATA}\"" ``` That happened because the prompt asked the model to manually place raw ES|QL inside a JSON string. For ES|QL containing backslashes and quotes, JSON serialization needs an additional escaping layer. The model-generated visualization looked correct visually, but after JSON parsing it no longer matched the authoritative ES|QL. ## Description This change ensures visualization configs preserve the ES|QL query produced by the graph. The prompt now embeds the ES|QL query as a JSON-encoded string, preventing raw interpolation from dropping escape levels for patterns like `GROK`. Before: <img width="639" height="854" alt="Screenshot 2026-06-03 at 14 25 37" src="https://github.com/user-attachments/assets/bd339e4a-57e6-4b5a-a1fd-6e7f606027a6" /> After: <img width="666" height="687" alt="Screenshot 2026-06-03 at 14 25 31" src="https://github.com/user-attachments/assets/c06d7dd0-862c-464a-9433-841422c9fbab" /> ## Test Plan - `node scripts/jest x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.ts` - `node scripts/eslint x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/prompts.ts` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
8 similar comments
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
…neration (elastic#272493) ## Summary Fixes ES|QL escaping drift during visualization config generation. ## What Was Not Working `generate_esql` produced the correct query for a GROK pattern: ```esql | GROK message "%{IP} - - \\[%{DATA}\\] \"%{WORD:method} %{DATA}\"" ``` But when the visualization config was generated, the query embedded in `visualization.data_source.query` lost one escaping level for the bracket regex: ```esql | GROK message "%{IP} - - \[%{DATA}\] \"%{WORD:method} %{DATA}\"" ``` That happened because the prompt asked the model to manually place raw ES|QL inside a JSON string. For ES|QL containing backslashes and quotes, JSON serialization needs an additional escaping layer. The model-generated visualization looked correct visually, but after JSON parsing it no longer matched the authoritative ES|QL. ## Description This change ensures visualization configs preserve the ES|QL query produced by the graph. The prompt now embeds the ES|QL query as a JSON-encoded string, preventing raw interpolation from dropping escape levels for patterns like `GROK`. Before: <img width="639" height="854" alt="Screenshot 2026-06-03 at 14 25 37" src="https://github.com/user-attachments/assets/bd339e4a-57e6-4b5a-a1fd-6e7f606027a6" /> After: <img width="666" height="687" alt="Screenshot 2026-06-03 at 14 25 31" src="https://github.com/user-attachments/assets/c06d7dd0-862c-464a-9433-841422c9fbab" /> ## Test Plan - `node scripts/jest x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.ts` - `node scripts/eslint x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/prompts.ts` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
Summary
Fixes ES|QL escaping drift during visualization config generation.
What Was Not Working
generate_esqlproduced the correct query for a GROK pattern:But when the visualization config was generated, the query embedded in
visualization.data_source.querylost one escaping level for the bracket regex:That happened because the prompt asked the model to manually place raw ES|QL inside a JSON string. For ES|QL containing backslashes and quotes, JSON serialization needs an additional escaping layer. The model-generated visualization looked correct visually, but after JSON parsing it no longer matched the authoritative ES|QL.
Description
This change ensures visualization configs preserve the ES|QL query produced by the graph. The prompt now embeds the ES|QL query as a JSON-encoded string, preventing raw interpolation from dropping escape levels for patterns like

GROK.Before:
After:

Test Plan
node scripts/jest x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.tsnode scripts/eslint x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/graph_lens.test.ts x-pack/platform/packages/shared/agent-builder/agent-builder-tools-base/visualization/prompts.tsChecklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesbackport:*labels.Identify risks
Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.