claude_code: adopt gen_ai.* fields and fix event.category assignments#19968
claude_code: adopt gen_ai.* fields and fix event.category assignments#19968efd6 wants to merge 2 commits into
Conversation
efd6
added
enhancement
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
This comment has been minimized.
This comment has been minimized.
TL;DRBuildkite failed on stale commit Remediation
Investigation detailsRoot CauseThe failed job ran At the failing commit, the newly added cleanup processor after - remove:
field: claude_code.events.duration_ms
ignore_missing: trueThat violates package-spec rule Evidence
Verification
What is this? | From workflow: PR Buildkite Detective Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. |
🚀 Benchmarks reportTo see the full report comment with |
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
![vera-review-bot[bot]](https://avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
| api_error: | ||
| category: | ||
| - web | ||
| - api |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/claude_code/data_stream/events/elasticsearch/ingest_pipeline/default.yml:104
api_error and api_retries_exhausted map event.type error under category api, but error is not a valid ECS type for api; use a valid type (e.g. info) and rely on event.outcome=failure.
Details
This change moves api_error and api_retries_exhausted from category web to api. The ECS spec lists the expected event.type values for category api as: access, admin, allowed, change, creation, deletion, denied, end, info, start, user. error is not among them (it was valid under the previous web category, whose expected types are access, error, info). The pair event.category: [api] + event.type: [error] is therefore no longer an ECS-sanctioned combination. It does not fail ingest or lint (both values are individually allowed), but it breaks ECS categorization semantics that detection rules and dashboards rely on. The failure signal is already carried by event.outcome, which the pipeline sets to failure for these events via set_event_outcome_failure.
Recommendation:
Keep the api category but use an ECS-valid type and let event.outcome convey the failure:
api_error:
category:
- api
type:
- info
api_retries_exhausted:
category:
- api
type:
- info🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| plugin_loaded: | ||
| category: | ||
| - package | ||
| - library |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/claude_code/data_stream/events/elasticsearch/ingest_pipeline/default.yml:149
plugin_loaded maps event.type info under category library, but library only allows type start in ECS; change the type to start (or revert the category).
Details
This change moves plugin_loaded from category package to library. Per the ECS spec, the only expected event.type value for category library is start; info is not valid for library (it was valid under the previous package category, whose expected types include info). The resulting pair event.category: [library] + event.type: [info] is not an ECS-sanctioned combination. This is a categorization-semantics regression, not an ingest/lint failure.
Recommendation:
Use the only ECS-valid type for the library category:
plugin_loaded:
category:
- library
type:
- start🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| ctx.event.category = ['iam']; | ||
| def decision = ctx.attributes?.decision; | ||
| if ('accept'.equals(decision)) { | ||
| ctx.event.type = ['allowed']; |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/claude_code/data_stream/events/elasticsearch/ingest_pipeline/default.yml:168
tool_decision sets event.type allowed/denied under category iam, but iam does not sanction those types in ECS; use type info and let event.outcome carry accept/reject.
Details
The new tool_decision branch sets event.category [iam] with event.type [allowed] (accept) or [denied] (reject). The ECS spec lists the expected event.type values for category iam as: admin, change, creation, deletion, group, info, user. Neither allowed nor denied is valid for iam (those types are expected only under network and intrusion_detection). The accept/reject distinction is already captured by event.outcome, which the pipeline sets to success/failure via set_event_outcome_decision_accept and set_event_outcome_decision_reject, so info is sufficient and ECS-valid for the type.
Recommendation:
Use an ECS-valid type for iam and rely on event.outcome for the accept/reject distinction:
if (name == 'tool_decision') {
ctx.event.category = ['iam'];
ctx.event.type = ['info'];
return;
}
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| - api | ||
| type: | ||
| - error | ||
| - info |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: low path: packages/claude_code/data_stream/events/elasticsearch/ingest_pipeline/default.yml:106
api_error and api_retries_exhausted now map event.type to info instead of error; consider keeping error so failed API events remain identifiable by event.type.
Details
This change re-categorizes api_error and api_retries_exhausted with event.type: [info] while event.outcome is still set to failure (via set_event_outcome_failure). error is a valid ECS event.type value intended exactly for events that describe an error, and it is the semantically accurate sub-bucket for these two event names. Typing an error event as info reduces its accuracy for the ECS categorization filters that detection rules and dashboards commonly use (e.g. event.category:api AND event.type:error); such consumers would no longer match these events even though they represent failures. The failure signal is preserved in event.outcome, so this is a minor semantic point rather than a functional break.
Recommendation:
Keep event.type: error for the two failure event types so they stay identifiable by categorization, consistent with the retained event.outcome: failure:
api_error:
category:
- api
type:
- error
api_retries_exhausted:
category:
- api
type:
- error🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
There was a problem hiding this comment.
This contradicts previous advice.
I agree that this is a loss, but the constraint exists. IMO, the api category should admit the error type.
andrewkroh
added
the
documentation
Map custom fields to their ECS gen_ai.* equivalents (model, tokens, cache, tool name/id/arguments, operation, provider) and to ECS core fields (error.code, error.type, organization.id, user.email, user.id). error.message is intentionally excluded due to elastic-package overloading the field. Fix event.category per event type: api_* events use "api" instead of "web"; tool_decision uses "iam" with allowed/denied type derived from the decision attribute; user_prompt and skill_activated drop their category (no closed-vocab value fits); hook_registered moves to "configuration"; hook_execution_start/complete get proper start/end types; plugin_loaded moves to "library". Derive event.outcome from the decision field for tool_decision events (accept=success, reject=failure) instead of leaving it unknown.
navnit-elastic
left a comment
navnit-elastic
left a comment
There was a problem hiding this comment.
Should we also update dashboards to use renamed fields?
|
@navnit-elastic, good catch. |
| "isBucketed": false, | ||
| "scale": "ratio", | ||
| "sourceField": "duration_ms", | ||
| "sourceField": "claude_code.events.duration_ms", |
There was a problem hiding this comment.
Severity: 🟠 High confidence: high path: packages/claude_code/kibana/dashboard/claude_code-cost-usage.json:1042
Panels reference claude_code.events.duration_ms, but this PR's pipeline removes that field (remove_ms_duration) after converting it to event.duration; point the panels at event.duration instead.
Details
The migration qualifies duration_ms to claude_code.events.duration_ms (lines 1042 and 1052), but in the same PR the ingest pipeline adds a remove processor (tag: remove_ms_duration) that deletes claude_code.events.duration_ms after copying it into event.duration (nanoseconds), and fields.yml drops the duration_ms field definition. No ingested document will carry claude_code.events.duration_ms, so these median/95th-percentile duration metrics will be empty.
Recommendation:
Reference the ECS field the pipeline populates instead. Note event.duration is in nanoseconds, so adjust the value format accordingly:
"sourceField": "event.duration"🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| "isBucketed": false, | ||
| "scale": "ratio", | ||
| "sourceField": "duration_ms", | ||
| "sourceField": "claude_code.events.duration_ms", |
There was a problem hiding this comment.
Severity: 🟠 High confidence: high path: packages/claude_code/kibana/dashboard/claude_code-mcp-server-access.json:1075
duration_ms median panel references claude_code.events.duration_ms, which this PR's pipeline removes; use event.duration instead.
Details
This panel's sourceField was migrated to claude_code.events.duration_ms, but the pipeline in this PR removes claude_code.events.duration_ms (tag: remove_ms_duration) after converting it to event.duration, and fields.yml no longer declares duration_ms. The panel will show no data.
Recommendation:
Point the metric at the populated ECS field (nanoseconds):
"sourceField": "event.duration"🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| "excludeIsRegex": false | ||
| }, | ||
| "sourceField": "duration_ms" | ||
| "sourceField": "claude_code.events.duration_ms" |
There was a problem hiding this comment.
Severity: 🟠 High confidence: high path: packages/claude_code/kibana/dashboard/claude_code-session-timeline.json:832
Session timeline references claude_code.events.duration_ms, which this PR's pipeline removes; use event.duration instead.
Details
The migrated sourceField claude_code.events.duration_ms is deleted by the pipeline's remove_ms_duration processor added in this PR (the value is preserved in event.duration, nanoseconds) and is no longer declared in fields.yml, so this table column will be empty.
Recommendation:
Reference the ECS field the pipeline populates (nanoseconds):
"sourceField": "event.duration"🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| "isBucketed": false, | ||
| "scale": "ratio", | ||
| "sourceField": "duration_ms", | ||
| "sourceField": "claude_code.events.duration_ms", |
There was a problem hiding this comment.
Severity: 🟠 High confidence: high path: packages/claude_code/kibana/dashboard/claude_code-tool-call-analysis.json:461
Tool-call-analysis duration panels reference claude_code.events.duration_ms, which this PR's pipeline removes; use event.duration instead.
Details
The migrated sourceField claude_code.events.duration_ms (also at lines 798, 808, and 1418) is deleted by the pipeline's remove_ms_duration processor added in this PR and dropped from fields.yml; the value now lives in event.duration (nanoseconds). These median/percentile duration metrics will be empty.
Recommendation:
Reference the populated ECS field (nanoseconds) for each occurrence:
"sourceField": "event.duration"🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| "type": "phrase", | ||
| "key": "tool_name", | ||
| "field": "tool_name", | ||
| "field": "gen_ai.tool.name", |
There was a problem hiding this comment.
Severity: 🟠 High confidence: medium path: packages/claude_code/kibana/dashboard/claude_code-mcp-server-access.json:437
MCP tool filters now query gen_ai.tool.name for the literal value "mcp_tool", but the pipeline maps mcp_tool_name (e.g. list_projects) into gen_ai.tool.name and removes the "mcp_tool" marker, so these filters match nothing.
Details
These filters (also at lines 794 and 989) were migrated from tool_name to gen_ai.tool.name while keeping the match value "mcp_tool". For MCP tool_result events the pipeline renames claude_code.events.mcp_tool_name into gen_ai.tool.name (yielding the specific tool name such as list_projects) and then removes the literal tool_name value "mcp_tool" (tag: remove_tool_name_after_mcp). gen_ai.tool.name therefore never equals "mcp_tool", so every panel gated on this filter returns zero results.
Recommendation:
Select MCP tool events by the presence of the MCP server name rather than the removed "mcp_tool" marker:
{
"meta": {
"type": "exists",
"key": "claude_code.events.mcp_server_name",
"field": "claude_code.events.mcp_server_name"
},
"query": { "exists": { "field": "claude_code.events.mcp_server_name" } }
}🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| }, | ||
| "4b22e68f-2839-cd9a-4737-d12438c65d2c": { | ||
| "label": "Top 20 values of mcp_tool_name", | ||
| "label": "Top 20 values of mcp_gen_ai.tool.name", |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: high path: packages/claude_code/kibana/dashboard/claude_code-mcp-server-access.json:1032
A global field rename mangled a column label into "Top 20 values of mcp_gen_ai.tool.name"; restore a clean label such as "MCP tool".
Details
The field-rename pass rewrote the substring inside the display label, producing "Top 20 values of mcp_gen_ai.tool.name". This is a cosmetic artifact of the search/replace, not a valid field path, and is shown to users as the column header.
Recommendation:
Give the column a readable label:
"label": "MCP tool"🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
|
✅ All changelog entries have the correct PR link. |
|
Reviewed the latest commits 826b0f5 — nothing new beyond already posted comments. Review summaryIssues found across earlier commits b71d1cb — 5 high, 1 low
Issues found across earlier commits 2baf602 — 1 low
Issues found across earlier commits 48134a5 — 3 medium
🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills
|
💚 Build Succeeded
History
cc @efd6 |
3 participants
Proposed commit message
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots