Skip to content

feat(gen-ai): add agent.node.* attributes for agent‑graph spans #2247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat(gen-ai): add agent.node.* attributes for agent‑graph spans #2247

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a27d7b4
feat(gen-ai): add agent.node.* attributes for multi‑step agent graphs
nadeemc2 May 11, 2025
6d461bd
feat(gen-ai): add agent.node.* attributes for multi‑step agent graphs
nadeemc2 May 11, 2025
36a67bf
Remove manually created docs and let them be auto-generated, added PR…
nadeemc2 May 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .chloggen/unreleased/agent-node.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the area of concern in the attributes-registry (e.g. http, cloud, db)
component: gen-ai

# A brief description of the change.
note: "Introduce `gen_ai.agent.node.type`, `gen_ai.agent.node.id`, and `gen_ai.agent.parent_id` attributes for fine‑grained agent‑graph spans."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't recommend defining attributes that are not referenced by any semantic conventions - https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/how-to-define-semantic-conventions.md#defining-attributes

Please define spans/metrics/events that would reference these attributes or add references to existing spans -


# Mandatory: tracking issues related to the change. Use the PR number if no separate issue exists.
issues: [2247] # replace with [12345] or similar when you know the issue/PR number

# (Optional) Additional information rendered under the primary note.
subtext: >
Enables node‑level latency, cost, and guard‑rail analysis for frameworks such
as CrewAI, LangGraph, and OpenAI Agents by standardising graph‑step metadata.
14 changes: 14 additions & 0 deletions docs/attributes-registry/gen-ai.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ This document defines the attributes used to describe telemetry in the context o
| <a id="gen-ai-agent-description" href="#gen-ai-agent-description">`gen_ai.agent.description`</a> | string | Free-form description of the GenAI agent provided by the application. | `Helps with math problems`; `Generates fiction stories` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-agent-id" href="#gen-ai-agent-id">`gen_ai.agent.id`</a> | string | The unique identifier of the GenAI agent. | `asst_5j66UpCpwteGg4YSxUnt7lPY` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-agent-name" href="#gen-ai-agent-name">`gen_ai.agent.name`</a> | string | Human-readable name of the GenAI agent provided by the application. | `Math Tutor`; `Fiction Writer` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-agent-node-id" href="#gen-ai-agent-node-id">`gen_ai.agent.node.id`</a> | string | Unique identifier of the node in the agent graph. | `step_3a87c9` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-agent-node-type" href="#gen-ai-agent-node-type">`gen_ai.agent.node.type`</a> | string | Kind of node within the agent execution graph. | `tool_call`; `decision`; `llm_call` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-agent-parent-id" href="#gen-ai-agent-parent-id">`gen_ai.agent.parent_id`</a> | string | Identifier of the node's direct parent in the graph (omit for root). | `step_1f62b0` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-conversation-id" href="#gen-ai-conversation-id">`gen_ai.conversation.id`</a> | string | The unique identifier for a conversation (session, thread), used to store and correlate messages within this conversation. | `conv_5j66UpCpwteGg4YSxUnt7lPY` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-data-source-id" href="#gen-ai-data-source-id">`gen_ai.data_source.id`</a> | string | The data source identifier. [1] | `H7STPQYOND` | ![Development](https://img.shields.io/badge/-development-blue) |
| <a id="gen-ai-operation-name" href="#gen-ai-operation-name">`gen_ai.operation.name`</a> | string | The name of the operation being performed. [2] | `chat`; `generate_content`; `text_completion` | ![Development](https://img.shields.io/badge/-development-blue) |
Expand Down Expand Up @@ -74,6 +77,17 @@ Datastore: A tool used by the agent to access and query structured or unstructur

---

`gen_ai.agent.node.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
|---|---|---|
| `decision` | A control‑flow / reasoning step | ![Development](https://img.shields.io/badge/-development-blue) |
| `llm_call` | A direct LLM invocation | ![Development](https://img.shields.io/badge/-development-blue) |
| `tool_call` | An external tool/function call | ![Development](https://img.shields.io/badge/-development-blue) |
| `vector_search` | Retrieval / vector‑store query step | ![Development](https://img.shields.io/badge/-development-blue) |

---

`gen_ai.operation.name` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
Expand Down
32 changes: 32 additions & 0 deletions model/gen-ai/registry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,38 @@ groups:
type: string
brief: Free-form description of the GenAI agent provided by the application.
examples: ["Helps with math problems", "Generates fiction stories"]
- id: gen_ai.agent.node.type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nadeemc2 , but can you share more detail for what does node maps for CrewAI, AutoGen, OpenAI SDK Agent? Thanks!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three frameworks ultimately execute graphs: vertices are “nodes”, edges are “what happens next”. The four enum values cover the common lowest‑level operations (model call, external function, branch, retriever), so any framework can map its richer taxonomy to them without losing fidelity. Instrumentations only need to look at the framework‑native step type and emit the corresponding node.type, rest of the Gen AI attributes (timing, cost, prompt‑id, etc.) work exactly as they do for plain LLM spans.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gyliu513 See some mapping examples below. Let me know, happy to share more details.

CrewAI:

  • llm_call → the moment a Task or Agent actually calls its configured LLM via Agent.run(); this is the basic step CrewAI records for every prompt/response 
  • tool_call → any invocation of a Tool (e.g., PythonTool, WebSearchTool) that CrewAI executes on behalf of the agent 
  • decision → the control‑flow checkpoints inside a Flow (if_, switch, or loop constructs) that decide which Task/Tool runs next 
  • vector_search → built‑in retrieval steps such as VectorSearchTool (or any RAG tool) that hit a vector store before the next LLM call

AG2:

  • llm_call → every time an autogen.Agent (e.g. AssistantAgent) executes generate_reply / a_generate_reply, which is where the framework actually calls the LLM.
  • tool_call → a function or code tool the agent runs when it was previously registered with register_function / Tool.
  • decision → the control step inside GroupChatManager (or swarm chat) that picks which agent will speak next; no LLM or tool is invoked here.
  • vector_search – a retrieval/RAG lookup executed by a retrieval‑capable agent (e.g. code‑execution retriever) before handing context to the LLM.

OpenAI SDK:

  • llm_call → each invocation of an Agent that reaches the LLM (the step that produces or updates the thread message)
  • tool_call → when the run executes (Runner) one of the agent’s registered tools (functions) before looping again
  • decision → the handoff step where the workflow picks a different agent (handoff in the run loop) without calling the LLM or any tool
  • vector_search → a retrieval/RAG tool run that fetches context before the next llm_call

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This attribute describes a type of inference call or another agent step type - it should be available as a part of GenAI span, but is not related to agents.

If it's set on the agent span, then we don't define agent-step-span - we only have invocation that covers multiple steps, so it won't fit there either.

If we define agent step span, it would be a duplicate of inference/tool call span under it. So I'm not sure what purpose this attribute solves and how we would populate it in the instrumentations.

stability: development
type:
members:
- id: llm_call
value: "llm_call"
brief: A direct LLM invocation
stability: development
- id: tool_call
value: "tool_call"
brief: An external tool/function call
stability: development
- id: decision
value: "decision"
brief: A control‑flow / reasoning step
stability: development
- id: vector_search
value: "vector_search"
brief: Retrieval / vector‑store query step
stability: development
brief: Kind of node within the agent execution graph.
examples: ["tool_call", "decision", "llm_call"]
- id: gen_ai.agent.node.id
stability: development
type: string
brief: Unique identifier of the node in the agent graph.
examples: ["step_3a87c9"]
- id: gen_ai.agent.parent_id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it gen_ai.agent.node.parent_id?

stability: development
type: string
brief: Identifier of the node's direct parent in the graph (omit for root).
examples: ["step_1f62b0"]
- id: gen_ai.tool.name
stability: development
type: string
Expand Down
Loading