[bot] CloudWeGo Eino integration doesn't trace Graph/Chain/Workflow runs or Retriever/Indexer/Loader/ChatTemplate components

[braintrust-bot]

What's missing

The trace/contrib/cloudwego/eino Handler only creates spans for three Eino component types: ChatModel, Tool, and Embedding. Its OnStart/OnStartWithStreamInput dispatch (trace/contrib/cloudwego/eino/traceeino.go:107-118, 272-300) calls model.ConvCallbackInput, tool.ConvCallbackInput, and embedding.ConvCallbackInput in sequence and returns ctx unchanged if none of them match — silently producing no span for anything else. OnEnd/OnError then no-op too, since they look up a span via spanKey{} in context and find none.

Two categories of upstream-supported instrumentation are dropped as a result:

1. Graph/Chain/Workflow-level run spans. Per Eino's own callback manual, the framework always triggers callbacks for the graph itself, distinct from its internal nodes: "Graph triggers Callback Handlers at its own start, end, and error timings" — via OnStart/OnEnd/OnError for Invoke, or the streaming variants for Stream/Collect/Transform. RunInfo.Component for this event is fixed to Graph / Chain / Workflow. Since the current handler doesn't recognize these, every multi-step Eino application — the primary use case of a graph/agent framework — produces zero top-level run span. Every ChatModel/Tool span instead appears as an unparented, flat sibling with no enclosing trace root, unlike this repo's Genkit and ADK integrations, which at least attempt a top-level run span for their equivalent orchestration primitives (see open issues on Genkit DefineFlow/ADK span nesting).

2. Retriever, Indexer, Document.Loader, Document.Transformer, ChatTemplate (Prompt) component spans. Eino's own callbacks.HandlerHelper documentation enumerates these as first-class, officially supported component types alongside ChatModel/Embedding/Tool: "HandlerHelper supports all official components. The current list is: ChatModel, ChatTemplate, Retriever, Indexer, Embedding, Document.Loader, Document.Transformer, Tool, ToolsNode." None of these five are handled here, so RAG-style Eino pipelines (retrieval, indexing, document loading/transforming, prompt templating) are invisible in Braintrust even though the callback data is available and typed.

This was previously partially audited: a prior pass found and fixed missing embeddings support and missing time_to_first_token for Eino (both now closed/fixed). This Graph/Chain/Retriever/Indexer/Loader/Transformer/ChatTemplate gap was not covered by that pass.

Braintrust docs status: not_found

https://www.braintrust.dev/docs/integrations/sdk-integrations/cloudwego-eino explicitly scopes "What Braintrust traces" to exactly three span types — chat model spans (eino.<model>), tool spans (eino.<tool>), and embedding spans (eino.<embedder>) — matching the current code. No mention of Graph/Chain/Workflow/Retriever/Indexer/Loader/Transformer/ChatTemplate spans anywhere on the page or in https://www.braintrust.dev/docs/instrument/trace-llm-calls. So this isn't docs drift — the docs accurately describe today's (narrower-than-upstream) behavior — but it is a real capability gap relative to what Eino's own callback framework guarantees and officially supports.

Upstream sources

• Eino callback manual — Graph self-triggering, RunInfo.Component fixed values, and HandlerHelper's official component list: https://www.cloudwego.io/docs/eino/core_modules/chain_and_graph_orchestration/callback_manual/
• components package (ComponentOfPrompt, ComponentOfRetriever, ComponentOfIndexer, ComponentOfLoader, ComponentOfTransformer, ComponentOfChatModel, ComponentOfEmbedding, ComponentOfTool): https://pkg.go.dev/github.com/cloudwego/eino/components
• callbacks package (RunInfo, TimingChecker): https://pkg.go.dev/github.com/cloudwego/eino/callbacks
• compose package (ComponentOfGraph, ComponentOfChain, ComponentOfWorkflow, ComponentOfLambda): https://pkg.go.dev/github.com/cloudwego/eino/compose

Braintrust docs inspected

• https://www.braintrust.dev/docs/integrations/sdk-integrations/cloudwego-eino
• https://www.braintrust.dev/docs/instrument/trace-llm-calls

Local repo files inspected

• trace/contrib/cloudwego/eino/traceeino.go (handler dispatch logic — OnStart, OnEnd, OnError, OnStartWithStreamInput, OnEndWithStreamOutput, Needed)
• trace/contrib/cloudwego/eino/traceeino_test.go
• trace/contrib/cloudwego/eino/integration_test.go (confirms only ChatModel/Tool/Embedding scenarios are exercised)
• trace/contrib/cloudwego/eino/orchestrion.go (confirms no separate auto-instrumentation path for graph-level spans)
• trace/contrib/README.md