Merge pull request #52 from agentevals-dev/docs/otel-recommendations

Update docs on OTel best practices

Author: krisztianfekete

docs/otel-compatibility.md

@@ -0,0 +1,90 @@
+# OpenTelemetry Compatibility
+
+agentevals consumes OpenTelemetry traces to evaluate AI agents. This document covers which OTel conventions we support, how we handle the ongoing migration from span events to log-based events, and guidance for instrumenting your own agents.
+
+## Supported Semantic Conventions
+
+### OTel GenAI Semantic Conventions (recommended)
+
+The [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) define standard span attributes for LLM interactions. agentevals auto-detects this format when spans contain `gen_ai.request.model` or `gen_ai.input.messages`.
+
+Supported attributes:
+
+| Attribute | Description |
+|-----------|-------------|
+| `gen_ai.request.model` | Model name (e.g. `gpt-4o`, `claude-sonnet-4-6`) |
+| `gen_ai.input.messages` | JSON array of input messages |
+| `gen_ai.output.messages` | JSON array of output messages |
+| `gen_ai.response.finish_reasons` | Why the model stopped generating |
+| `gen_ai.usage.input_tokens` | Input token count |
+| `gen_ai.usage.output_tokens` | Output token count |
+| `gen_ai.system` | AI system identifier (e.g. `openai`, `anthropic`) |
+
+This format works with LangChain, Strands, OpenAI instrumentation, Anthropic instrumentation, and any framework that follows the GenAI semantic conventions.
+
+### Google ADK (framework-native)
+
+Google ADK emits spans under the `gcp.vertex.agent` OTel scope with proprietary attributes (`gcp.vertex.agent.llm_request`, `gcp.vertex.agent.llm_response`, etc.). agentevals has a dedicated converter that auto-detects this format. No GenAI semconv configuration is needed.
+
+### Format Detection
+
+Format detection is automatic. When a trace contains both ADK and GenAI attributes, ADK takes priority because it provides richer structured data. The detection logic lives in `src/agentevals/converter.py` (`get_extractor()`).
+
+## Message Content Delivery
+
+GenAI message content (`gen_ai.input.messages`, `gen_ai.output.messages`) can arrive through three mechanisms. agentevals supports all of them:
+
+### 1. Span attributes (simplest)
+
+Message content is stored directly as span attributes. This is the most straightforward approach and requires no special handling.
+
+### 2. Log records (recommended for new instrumentation)
+
+Message content is emitted as OTel log records correlated with spans via trace context. This is the pattern used by `opentelemetry-instrumentation-openai-v2` and LangChain's GenAI instrumentation.
+
+Requires both `OTLPSpanExporter` and `OTLPLogExporter` (or their streaming equivalents). Set `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` to enable content capture.
+
+### 3. Span events (deprecated, supported for backward compatibility)
+
+Message content is emitted as attributes on span events. agentevals promotes these to span-level attributes during normalization so downstream processing sees a uniform shape.
+
+This promotion happens in three processing layers:
+- `streaming/processor.py` for live WebSocket spans
+- `api/otlp_routes.py` for OTLP HTTP reception
+- `loader/otlp.py` for loading OTLP JSON files
+
+## Span Events Deprecation
+
+The OTel community is [deprecating the Span Event API](https://opentelemetry.io/blog/2026/deprecating-span-events/) (`Span.AddEvent`, `Span.RecordException`) in favor of emitting events as log records via the Logs API. The core idea: "events are logs with names," correlated with traces through context.
+
+### What this means for agentevals users
+
+**No immediate action required.** Existing instrumentation continues to work. The deprecation is about providing a single recommended path for new code, not about removing support for existing span event data.
+
+**For new instrumentation**, prefer the logs-based pattern. Configure both `OTLPSpanExporter` and `OTLPLogExporter`, and use instrumentation libraries that emit message content as log records.
+
+**For existing span-event instrumentation** (e.g. Strands with `OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental`), everything continues to work. When your framework releases a version that migrates to log-based events, update your exporter configuration to include `OTLPLogExporter` and follow the logs-based pattern.
+
+### What this means for agentevals internals
+
+agentevals already supports both content delivery mechanisms. The span event promotion logic will remain for backward compatibility with older instrumentation versions. As frameworks migrate, the log-based path (already fully supported) will become the primary path.
+
+### Migration checklist for framework authors
+
+If you maintain an OTel-instrumented agent framework and want to align with the deprecation:
+
+1. Emit `gen_ai.input.messages` and `gen_ai.output.messages` as log records instead of span events
+2. Correlate logs with spans via trace context (the OTel SDK handles this automatically)
+3. Document that users need both `OTLPSpanExporter` and `OTLPLogExporter`
+4. Consider an opt-in flag (similar to `OTEL_SEMCONV_EXCEPTION_SIGNAL_OPT_IN`) during the transition
+
+## OTLP Receiver
+
+agentevals runs an OTLP HTTP receiver on port 4318 (the standard OTLP HTTP port) that accepts:
+
+| Endpoint | Content Types |
+|----------|--------------|
+| `/v1/traces` | `application/json`, `application/x-protobuf` |
+| `/v1/logs` | `application/json`, `application/x-protobuf` |
+
+Point your standard OTel exporters at `http://localhost:4318` and traces will stream into agentevals automatically. See [examples/README.md](../examples/README.md) for zero-code setup instructions.

docs/streaming.md

@@ -106,6 +106,22 @@ When a session ends (`session_end` message), the server:
 
 Evaluation is triggered separately from the UI or API.
 
+## GenAI Message Content: Span Events vs. Logs
+
+agentevals supports two mechanisms for receiving GenAI message content (`gen_ai.input.messages`, `gen_ai.output.messages`):
+
+**Log records (recommended).** Instrumentation libraries like `opentelemetry-instrumentation-openai-v2` emit message content as OTel log records correlated with spans via trace context. The server merges these back into spans during session completion (see `log_enrichment.py`).
+
+**Span events (legacy, supported for backward compatibility).** Some frameworks (notably Strands with `OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental`) currently emit message content as span event attributes. The `AgentEvalsStreamingProcessor` promotes these attributes to span-level attributes so downstream converters see a uniform shape. This promotion happens in three places:
+
+- `streaming/processor.py` for live WebSocket spans
+- `api/otlp_routes.py` for OTLP HTTP reception
+- `loader/otlp.py` for loading OTLP JSON files
+
+The OTel community is [deprecating span events](https://opentelemetry.io/blog/2026/deprecating-span-events/) in favor of log-based events emitted via the Logs API. As frameworks migrate, the log-based path will become the standard. The span event promotion logic will remain for backward compatibility with older instrumentation versions.
+
+For a full overview of supported OTel conventions and migration guidance, see [otel-compatibility.md](./otel-compatibility.md).
+
 ## WebSocket Protocol
 
 ### Endpoint: `/ws/traces`

examples/README.md

@@ -92,12 +92,14 @@ Detection checks for `gen_ai.request.model` / `gen_ai.input.messages` (GenAI sem
 | Example | Framework | LLM Provider | Instrumentation | Content Delivery |
 |---------|-----------|-------------|-----------------|-----------------|
 | [zero-code-examples/langchain/](./zero-code-examples/langchain/) | LangChain | OpenAI | GenAI semconv (logs) | Standard OTLP export |
-| [zero-code-examples/strands/](./zero-code-examples/strands/) | Strands | OpenAI | GenAI semconv (events) | Standard OTLP export |
+| [zero-code-examples/strands/](./zero-code-examples/strands/) | Strands | OpenAI | GenAI semconv (events*) | Standard OTLP export |
 | [zero-code-examples/adk/](./zero-code-examples/adk/) | Google ADK | Gemini | ADK built-in | Standard OTLP export |
 | [langchain_agent](./langchain_agent/) | LangChain | OpenAI | GenAI semconv (logs) | SDK WebSocket |
-| [strands_agent](./strands_agent/) | Strands | OpenAI | GenAI semconv (events) | SDK WebSocket |
+| [strands_agent](./strands_agent/) | Strands | OpenAI | GenAI semconv (events*) | SDK WebSocket |
 | [dice_agent](./dice_agent/) | Google ADK | Gemini | ADK built-in | SDK WebSocket |
 
+*\*Span events are [being deprecated](https://opentelemetry.io/blog/2026/deprecating-span-events/) in favor of log-based events. agentevals supports both. See [docs/otel-compatibility.md](../docs/otel-compatibility.md) for details.*
+
 The zero-code and SDK examples implement the same toy agent (dice rolling + prime checking) so you can compare the two approaches directly.
 
 ## Advanced: GenAI Semantic Convention Patterns
@@ -135,6 +137,9 @@ See [langchain_agent/README.md](./langchain_agent/README.md) for the full walkth
 
 ### Events-Based Content ([strands_agent](./strands_agent/))
 
+> [!NOTE]
+> The OTel community is [deprecating span events](https://opentelemetry.io/blog/2026/deprecating-span-events/) in favor of log-based events emitted via the Logs API. Frameworks currently using span events (like Strands) are expected to migrate to log-based events in future versions. agentevals supports both patterns and will continue to handle span events for backward compatibility.
+
 Used by frameworks that emit message content as **span events** rather than separate log records. The `AgentEvalsStreamingProcessor` automatically promotes `gen_ai.input.messages` and `gen_ai.output.messages` from event attributes to span attributes, so downstream processing sees a uniform shape.
 
 This pattern needs only a `TracerProvider`, no `LoggerProvider` or log processor:
@@ -149,11 +154,14 @@ telemetry.tracer_provider.add_span_processor(processor)
 
 ### Which Pattern Should I Use?
 
+- **For new instrumentation, prefer the logs-based pattern.** The OTel community recommends emitting events as log records rather than span events going forward.
 - **Check your framework/library docs first.** They will tell you whether message content is emitted as logs or span events.
 - If your instrumentation library requires a `LoggerProvider` (like `opentelemetry-instrumentation-openai-v2`), use the **logs-based** pattern.
-- If your framework emits GenAI span events (like Strands with `StrandsTelemetry`), use the **events-based** pattern, it's simpler.
+- If your framework currently emits GenAI span events (like Strands with `StrandsTelemetry`), the **events-based** pattern works today. When the framework migrates to log-based events, switch to the logs-based pattern.
 - If you're using **Google ADK**, skip GenAI semconv entirely. See the next section.
 
+For a detailed overview of OTel compatibility and the ongoing migration, see [docs/otel-compatibility.md](../docs/otel-compatibility.md).
+
 ## Framework-Native Tracing (Google ADK)
 
 Google ADK instruments agents automatically under the `gcp.vertex.agent` OTel scope. It emits proprietary attributes (`gcp.vertex.agent.llm_request`, `gcp.vertex.agent.llm_response`, etc.) directly on spans. agentevals has a dedicated converter for this format.

examples/strands_agent/main.py

@@ -10,6 +10,12 @@
    events with gen_ai.input.messages / gen_ai.output.messages attributes, which the
    streaming processor promotes from span events to span attributes
 
+Note: Strands currently delivers message content via span events. The OTel community
+is deprecating span events in favor of log-based events (see
+https://opentelemetry.io/blog/2026/deprecating-span-events/). Future Strands versions
+will likely migrate to log-based events, at which point this example should switch to
+the logs-based pattern used in langchain_agent/.
+
 Prerequisites:
     1. Install dependencies:
        $ pip install -r requirements.txt