Otel memory instrumentation

[henrikrexed]

📝 Pull Request Summary

Adds OpenTelemetry instrumentation to memU's memory operations (span/attribute names following memory-semconv v0.1.0) and W3C trace-context propagation across the agent→memU process boundary, so a single memu CLI invocation joins the caller's distributed trace end-to-end. Includes a real CrewAI + Ollama end-to-end driver that produces one continuous agent → memory trace (the memU analog of the Memobase CrewAI demo).

---

✅ What does this PR do?

• New memu.observability package (src/memu/observability/): a semconv naming contract, memory_operation / traced_embed instrumentation, an OTLP SDK wiring layer (init_telemetry), a config gating the SDK, and providers/instruments for metrics.
• W3C propagation across the process boundary (propagation.py): extract_context_from_env() parents memU's spans under the caller's TRACEPARENT/TRACESTATE; env_with_current_context() lets an agent/host adapter inject its active trace into a memu subprocess. Because memU is CLI/subprocess-invoked (no in-process HTTP server), the env-var carrier is the OTel-idiomatic choice.
• CLI SERVER span (entrypoint.py + cli.py): each memu invocation is wrapped in one memu.<command> SERVER span parented to the inbound trace, so the memory.* spans it emits nest into one end-to-end agent → memory trace instead of standing alone. Flushes on exit for short-lived CLI processes.
• AgenticMixin instrumentation (app/agentic.py): commit/retrieve/update/delete and embedding are wrapped in memory.write / memory.read / memory.embed INTERNAL spans carrying memory.operation, memory.query.length, memory.input.size_bytes, memory.top_similarity, memory.results.count. PII-safe: emits query length, not query text (toggleable via config).
• Optional observability extra (pyproject.toml): hard-depends on opentelemetry-api only (no-op without an SDK); the SDK + OTLP gRPC exporter live in the [observability] extra.
• CrewAI + Ollama e2e + validation scripts (scripts/): otel_e2e_crewai_agent.py drives a real CrewAI crew (LLM = Ollama on Mac Studio) whose memory read/write goes through the instrumented memU; plus otel_e2e_memory_server.py, otel_e2e_trace_render.py, otel_validation_run.py, and otel_validate_*.sh. README-otel-e2e-crewai.md documents the pinned environment and the resulting span tree.

Resulting trace (two services, one trace_id):

---

🤔 Why is this change needed?

• memU previously emitted no telemetry: memory operations were invisible to observability backends, and an agent's trace and its memu calls landed as disconnected single-op traces.
• This makes agent → memory latency, recall quality, and embedding cost observable in one span tree, with memory-specific semantic attributes that let a backend (e.g. Dynatrace/Grail) reason about memory behavior specifically.
• Driven by a real end-to-end CrewAI + Ollama demonstration as requested.

---

🔍 Type of Change

• Bug fix
• New feature
• Documentation update
• Refactor / cleanup
• Other (please explain)

---

✅ PR Quality Checklist

• PR title follows an allowed format (feat(observability): …)
• Changes are limited in scope and easy to review
• Documentation updated where applicable (scripts/README-otel-e2e-crewai.md)
• No breaking changes (instrumentation is a no-op unless an OTLP endpoint is configured; only adds opentelemetry-api as a hard dep and an opt-in observability extra)
• Related issues or discussions linked

---

📌 Optional

• Screenshots or examples added
• Edge cases considered (no-traceparent → fresh root trace; SDK absent → no-op; PII: query length not text)
• Follow-up tasks mentioned (CrewAI demo scripts are reproducible harnesses, not shipped runtime code)

Verification

• pytest tests/test_observability.py → 31 passed (context extraction/injection, search span joining the inbound trace, CLI SERVER span under an agent trace, no-op-when-disabled).
• Real run: scripts/otel_e2e_crewai_agent.py against Ollama (llama4:scout) + memU → one trace_id across both services, confirmed in Dynatrace via DQL.

[Copilot]

Pull request overview

This PR introduces an OpenTelemetry-based observability layer for memU’s memory operations, including W3C trace-context propagation across the agent → memu CLI subprocess boundary, plus in-process tests validating spans/metrics/logs against the proposed memory-semconv v0.1.0 contract.

Changes:

• Add a new memu.observability package (semconv constants, span/metric/log helpers, OTLP SDK bootstrap, and env-based trace-context propagation).
• Instrument the CLI entrypoint with a SERVER span and instrument AgenticMixin read/write/embed paths with memory.* spans and metrics.
• Add a comprehensive in-process test suite validating emitted telemetry.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
uv.lock	Adds OTel-related locked dependencies and an observability extra resolution.
pyproject.toml	Adds opentelemetry-api as a base dependency and an observability extra for SDK/exporters; adds a ruff ignore for semconv constants.
src/memu/cli.py	Wraps each CLI invocation in cli_telemetry(...) for a top-level SERVER span.
src/memu/app/agentic.py	Wraps list/retrieve/commit and embed calls in memory-semconv spans and records metrics.
src/memu/observability/init.py	Exposes observability public API surface.
src/memu/observability/config.py	Adds env-driven enablement + PII gate for query text + store-backend override.
src/memu/observability/entrypoint.py	Implements CLI-level SERVER span + lifecycle wiring.
src/memu/observability/instruments.py	Defines metric instruments and label-cardinality guardrails.
src/memu/observability/operation.py	Implements per-operation span/metric/log lifecycle and traced_embed.
src/memu/observability/propagation.py	Implements env-var carrier extraction/injection for W3C trace-context.
src/memu/observability/providers.py	Adds a provider registry to avoid reliance on OTel process globals.
src/memu/observability/semconv.py	Centralizes span/attribute/metric names for memory-semconv v0.1.0.
src/memu/observability/telemetry.py	Implements SDK bootstrap and OTLP wiring helpers.
tests/test_observability.py	Adds in-process tests validating spans/metrics/logs and trace joining across boundaries.

Suppressed comments (2)

src/memu/observability/telemetry.py:70

• build_resource() will currently crash with an AttributeError if it’s called without the OpenTelemetry SDK installed (after guarding the module import). Since it’s a public API (__all__), it should fail fast with a clear install hint when the SDK isn’t available.

def build_resource(resource_attributes: Mapping[str, str] | None, store_backend: str | None) -> Resource:
    """Assemble the per-process resource with the ``memory.sut.*`` identity."""
    attrs: dict[str, Any] = {
        SERVICE_NAME: semconv.SUT_NAME,

src/memu/observability/telemetry.py:116

• init_telemetry() should explicitly reject calls when opentelemetry-sdk isn’t installed, instead of failing later with confusing attribute/NameErrors. This also matches the stated contract that the SDK is optional and only required when you actually wire/export telemetry.

def init_telemetry(
    *,
    resource_attributes: Mapping[str, str] | None = None,
    store_backend: str | None = None,
    span_exporter: SpanExporter | None = None,
    metric_reader: MetricReader | None = None,
    log_processor: LogRecordProcessor | None = None,
    set_global: bool = True,
    force: bool = False,
) -> TelemetryHandle:

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[xnne-bot]

Thanks — solid work, and the CrewAI + Ollama e2e is much appreciated.

That said, we can't take this in as-is. memU's telemetry direction is being set by #613, which routes CLI analytics and error reporting to the memu-service backend API rather than OTLP — and that work lives in a private backend repo, so it's not something external contributors can meaningfully participate in. Merging a parallel OTel path now would put us on two telemetry surfaces to maintain.

The observability extra is opt-in, so the instrumentation can still live on as a fork/patch if you'd like to keep using it.