Skip to content

Added kafkaQueueProblems scenarios #269901

Merged
yuliia-fryshko merged 1 commit into
elastic:9.4from
yuliia-fryshko:backport/9.4/pr-267402
May 19, 2026
Merged

Added kafkaQueueProblems scenarios #269901
yuliia-fryshko merged 1 commit into
elastic:9.4from
yuliia-fryshko:backport/9.4/pr-267402

Conversation

@yuliia-fryshko
Copy link
Copy Markdown
Contributor

Backport

This will backport the following commits from main to 9.4:

Added kafkaQueueProblems scenarios #267402

Questions ?

Please refer to the Backport tool documentation

Closes elastic/obs-ai-team#436

This PR adds AI Insight Feedback Telemetry:
- Added FeedbackButtons component with Yes/No buttons for user feedback
- Created telemetry event schema
(`observability_agent_builder_ai_insight_feedback`) tracking feedback
and insight_type
- For the "Start Conversation" button we will use kibana-browse events.
For this we only had to add
`data-test-subj=obsAgentBuilder-{log|alert|error}-startConversationButton`
to the ui component

**Test plan**

-  run kibana locally;
- Open and Click to expand  any AI Insight
- Wait for the AI summary to be fully generated
- Verify feedback buttons appear
- Confirm the "Was this helpful?" text is displayed with Yes and No
buttons
- Click either Yes or No button
- Verify:
- A toast notification appears with "Thanks for your feedback!"
- The clicked button becomes disabled

Verify telemetry event in Dev Tools:
- Open Kibana Dev Tools
- Run the following query:

```
GET ebt-kibana-browser/_search
{
  "query": {
    "match": {
      "event_type": "observability_agent_builder_ai_insight_feedback"
    }
  }
}

```

- Validate event payload and confirm the response contains a hit with:
```
 "event_type": "observability_agent_builder_ai_insight_feedback",
"properties.feedback": "positive" or "negative"
"properties.insight_type": "log", "alert", or "error"
```

- click Start Conversation button;
- Open Kibana Dev Tools
- Run the following query:
```
GET ebt-kibana-browser/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "event_type": "click" }},
        { "term": { "properties.target": "data-test-subj=obsAgentBuilder-log-startConversationButton" }}
      ]
    }
  }
}
```

<img width="1142" height="284" alt="Screenshot 2026-01-30 at 19 17 21"
src="https://github.com/user-attachments/assets/27912cce-17eb-4e33-bb2d-957ceb64e920"
/>
<img width="1163" height="97" alt="Screenshot 2026-01-30 at 19 24 59"
src="https://github.com/user-attachments/assets/7acced69-9683-48f6-bbd5-e40088766ab4"
/>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Viduni Wickramarachchi <viduni.ushanka@gmail.com>
(cherry picked from commit 4b56723)
@yuliia-fryshko yuliia-fryshko requested a review from a team May 19, 2026 10:52
@yuliia-fryshko yuliia-fryshko self-assigned this May 19, 2026
@yuliia-fryshko yuliia-fryshko added backport This PR is a backport of another PR evals:observability-ai Run the observability-ai evals @kbn/evals models:judge:eis/google-gemini-3.1-pro Override LLM-as-a-judge connector for evals: eis/google-gemini-3.1-pro models:weekly-eis-models Run evals against the weekly EIS model set (see eval_pipeline.ts) labels May 19, 2026
@kibanamachine kibanamachine requested review from viduni94 and removed request for a team and kibanamachine May 19, 2026 10:52
@kibanamachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #55 / Entity Analytics - Watchlists @ess @serverless @skipInServerlessMKI Entity Store Attribute Sync should remove the watchlist id from entity.attributes.watchlists on deletion but preserve other ids

Metrics [docs]

✅ unchanged

cc @yuliia-fryshko

@yuliia-fryshko yuliia-fryshko merged commit e6478c5 into elastic:9.4 May 19, 2026
86 checks passed
@yuliia-fryshko yuliia-fryshko changed the title Add AI Insight Feedback Telemetry (#251124) Added kafkaQueueProblems scenarios #267402 May 19, 2026
@yuliia-fryshko yuliia-fryshko changed the title Added kafkaQueueProblems scenarios #267402 Added kafkaQueueProblems scenarios May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR evals:observability-ai Run the observability-ai evals @kbn/evals models:judge:eis/google-gemini-3.1-pro Override LLM-as-a-judge connector for evals: eis/google-gemini-3.1-pro models:weekly-eis-models Run evals against the weekly EIS model set (see eval_pipeline.ts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants