Skip to content

feat(gen-ai): add agent.node.* attributes for agent‑graph spans #2247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

nadeemc2
Copy link

Changes

This PR adds node‑level attributes to the Gen AI semantic conventions so agent frameworks (CrewAI, LangGraph, AutoGen v2, OpenAI Agents SDK, etc.) can emit spans for each step in an execution graph.

  • New attributes
    • gen_ai.agent.node.type – enum: llm_call, tool_call, decision, vector_search.
    • gen_ai.agent.node.id – stable identifier for each node.
    • gen_ai.agent.parent_id – direct parent node ID (blank for roots).
  • Registry updates
    • Added definitions to model/gen-ai/registry.yaml using type.members enum syntax.
  • Documentation
    • Regenerated docs/attributes-registry/gen-ai.md.
    • Added new explanatory page docs/gen-ai/agent-node-spans.md.
  • Ran make fix; all markdown is autogenerated.

Rationale

Current Gen AI spans capture model calls but cannot express the topology of multi‑step agents (decisions, tool calls, RAG look‑ups).
Standardising agent.node.* unlocks:

  • Per‑step latency and cost analysis.
  • Prompt‑injection/guard‑rail tracing at the node level.
  • Vendor‑neutral graph visualisation across frameworks.

Impact

  • Backwards‑compatible – additive only.
  • Language generators will expose new constants; no existing code breaks.
  • Instrumentations can start emitting the fields immediately.

Merge requirement checklist

  • CONTRIBUTING.md guidelines followed.
  • Changelog fragment added in .chloggen/unreleased/.
  • schema-next.yaml NOT changed (new attributes only).

Copy link

linux-foundation-easycla bot commented May 11, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

@nadeemc2 nadeemc2 force-pushed the feat/agent-node-spans branch from 2535037 to 6d461bd Compare May 11, 2025 23:35
@@ -231,6 +231,38 @@ groups:
type: string
brief: Free-form description of the GenAI agent provided by the application.
examples: ["Helps with math problems", "Generates fiction stories"]
- id: gen_ai.agent.node.type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nadeemc2 , but can you share more detail for what does node maps for CrewAI, AutoGen, OpenAI SDK Agent? Thanks!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three frameworks ultimately execute graphs: vertices are “nodes”, edges are “what happens next”. The four enum values cover the common lowest‑level operations (model call, external function, branch, retriever), so any framework can map its richer taxonomy to them without losing fidelity. Instrumentations only need to look at the framework‑native step type and emit the corresponding node.type, rest of the Gen AI attributes (timing, cost, prompt‑id, etc.) work exactly as they do for plain LLM spans.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gyliu513 See some mapping examples below. Let me know, happy to share more details.

CrewAI:

  • llm_call → the moment a Task or Agent actually calls its configured LLM via Agent.run(); this is the basic step CrewAI records for every prompt/response 
  • tool_call → any invocation of a Tool (e.g., PythonTool, WebSearchTool) that CrewAI executes on behalf of the agent 
  • decision → the control‑flow checkpoints inside a Flow (if_, switch, or loop constructs) that decide which Task/Tool runs next 
  • vector_search → built‑in retrieval steps such as VectorSearchTool (or any RAG tool) that hit a vector store before the next LLM call

AG2:

  • llm_call → every time an autogen.Agent (e.g. AssistantAgent) executes generate_reply / a_generate_reply, which is where the framework actually calls the LLM.
  • tool_call → a function or code tool the agent runs when it was previously registered with register_function / Tool.
  • decision → the control step inside GroupChatManager (or swarm chat) that picks which agent will speak next; no LLM or tool is invoked here.
  • vector_search – a retrieval/RAG lookup executed by a retrieval‑capable agent (e.g. code‑execution retriever) before handing context to the LLM.

OpenAI SDK:

  • llm_call → each invocation of an Agent that reaches the LLM (the step that produces or updates the thread message)
  • tool_call → when the run executes (Runner) one of the agent’s registered tools (functions) before looping again
  • decision → the handoff step where the workflow picks a different agent (handoff in the run loop) without calling the LLM or any tool
  • vector_search → a retrieval/RAG tool run that fetches context before the next llm_call

Copy link
Member

@gyliu513 gyliu513 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if node is a good concept or not, but we can get more comments from others

type: string
brief: Unique identifier of the node in the agent graph.
examples: ["step_3a87c9"]
- id: gen_ai.agent.parent_id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it gen_ai.agent.node.parent_id?

component: gen-ai

# A brief description of the change.
note: "Introduce `gen_ai.agent.node.type`, `gen_ai.agent.node.id`, and `gen_ai.agent.parent_id` attributes for fine‑grained agent‑graph spans."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't recommend defining attributes that are not referenced by any semantic conventions - https://github.com/open-telemetry/semantic-conventions/blob/main/docs/general/how-to-define-semantic-conventions.md#defining-attributes

Please define spans/metrics/events that would reference these attributes or add references to existing spans -

@@ -231,6 +231,38 @@ groups:
type: string
brief: Free-form description of the GenAI agent provided by the application.
examples: ["Helps with math problems", "Generates fiction stories"]
- id: gen_ai.agent.node.type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This attribute describes a type of inference call or another agent step type - it should be available as a part of GenAI span, but is not related to agents.

If it's set on the agent span, then we don't define agent-step-span - we only have invocation that covers multiple steps, so it won't fit there either.

If we define agent step span, it would be a duplicate of inference/tool call span under it. So I'm not sure what purpose this attribute solves and how we would populate it in the instrumentations.

@github-project-automation github-project-automation bot moved this from Untriaged to Blocked in Semantic Conventions Triage May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

3 participants