Skip to content

feat: add OpenTelemetry distributed tracing#21

Merged
Siddhant-K-code merged 2 commits into
mainfrom
feat/opentelemetry-tracing
Feb 15, 2026
Merged

feat: add OpenTelemetry distributed tracing#21
Siddhant-K-code merged 2 commits into
mainfrom
feat/opentelemetry-tracing

Conversation

@Siddhant-K-code
Copy link
Copy Markdown
Owner

Motivation

Prometheus metrics (#7) cover aggregate counters and histograms. Distributed tracing adds per-request visibility — see exactly how long each pipeline stage takes and where bottlenecks are.

Closes #9

Changes

  • pkg/telemetry/ — OTEL provider init with OTLP, stdout, and no-op exporters. Span helpers for each pipeline stage. W3C Trace Context + Baggage propagation. Configurable sampling rate.
  • cmd/api.go — Root span for /v1/dedupe, child spans for embedding, clustering, selection, MMR. Error recording on failures. Result attributes on root span.
  • cmd/serve.go — Root span for /v1/retrieve with result attributes and error recording.
  • pkg/config/config.goTelemetryConfig and TracingConfig structs, defaults, validation (exporter, sample_rate), env interpolation, updated YAML template.
  • README.md — Tracing section with config examples, span table, env var usage.

Spans per request

distill.request (root)
├── distill.embedding      (if text chunks need embedding)
├── distill.clustering     (agglomerative clustering)
├── distill.selection      (representative selection)
└── distill.mmr            (MMR re-ranking, if targetK set)

Config

telemetry:
  tracing:
    enabled: true
    exporter: otlp
    endpoint: localhost:4317
    sample_rate: 1.0
    insecure: true

Testing

  • 13 unit tests: disabled/enabled init, all exporters, invalid exporter, sample rate, shutdown, all span helpers, RecordResult, RecordError, zero-input edge case
  • All existing tests pass
  • go vet clean

Add OTEL tracing with per-stage spans across the dedup pipeline:
- pkg/telemetry: provider init, span helpers for each pipeline stage,
  OTLP/stdout/none exporters, W3C Trace Context propagation
- Instrumented api.go (embedding, clustering, selection, MMR spans)
- Instrumented serve.go (retrieval root span with result attributes)
- Added TelemetryConfig to pkg/config with validation
- Updated distill.yaml template with telemetry.tracing section
- 13 unit tests for telemetry package
- README tracing documentation with span table

Closes #9

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code added the enhancement New feature or request label Feb 15, 2026
Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code merged commit 4f05212 into main Feb 15, 2026
2 checks passed
@Siddhant-K-code Siddhant-K-code deleted the feat/opentelemetry-tracing branch February 15, 2026 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] OpenTelemetry Tracing

1 participant