feat: add OpenTelemetry distributed tracing#21
Merged
Conversation
Add OTEL tracing with per-stage spans across the dedup pipeline: - pkg/telemetry: provider init, span helpers for each pipeline stage, OTLP/stdout/none exporters, W3C Trace Context propagation - Instrumented api.go (embedding, clustering, selection, MMR spans) - Instrumented serve.go (retrieval root span with result attributes) - Added TelemetryConfig to pkg/config with validation - Updated distill.yaml template with telemetry.tracing section - 13 unit tests for telemetry package - README tracing documentation with span table Closes #9 Co-authored-by: Ona <no-reply@ona.com>
Co-authored-by: Ona <no-reply@ona.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Prometheus metrics (#7) cover aggregate counters and histograms. Distributed tracing adds per-request visibility — see exactly how long each pipeline stage takes and where bottlenecks are.
Closes #9
Changes
pkg/telemetry/— OTEL provider init with OTLP, stdout, and no-op exporters. Span helpers for each pipeline stage. W3C Trace Context + Baggage propagation. Configurable sampling rate.cmd/api.go— Root span for/v1/dedupe, child spans for embedding, clustering, selection, MMR. Error recording on failures. Result attributes on root span.cmd/serve.go— Root span for/v1/retrievewith result attributes and error recording.pkg/config/config.go—TelemetryConfigandTracingConfigstructs, defaults, validation (exporter, sample_rate), env interpolation, updated YAML template.README.md— Tracing section with config examples, span table, env var usage.Spans per request
Config
Testing
go vetclean