|
| 1 | +# Distributed Tracing Configuration |
| 2 | + |
| 3 | +Nantian Gateway supports OpenTelemetry distributed tracing via OTLP (gRPC). |
| 4 | + |
| 5 | +## Control Plane |
| 6 | + |
| 7 | +Enable tracing in the control plane config: |
| 8 | + |
| 9 | +```yaml |
| 10 | +tracing: |
| 11 | + enabled: true |
| 12 | + endpoint: "otel-collector:4317" |
| 13 | + insecure: true |
| 14 | + samplerRatio: 0.1 |
| 15 | + headers: |
| 16 | + x-api-key: "my-key" |
| 17 | +``` |
| 18 | +
|
| 19 | +| Field | Type | Default | Description | |
| 20 | +|---|---|---|---| |
| 21 | +| `enabled` | bool | `false` | Enable OTLP tracing | |
| 22 | +| `endpoint` | string | — | OTLP gRPC collector endpoint | |
| 23 | +| `insecure` | bool | `false` | Use plaintext (no TLS) | |
| 24 | +| `samplerRatio` | float | `1.0` | Sampling ratio (0.0—1.0) | |
| 25 | +| `headers` | map | — | Custom gRPC metadata headers | |
| 26 | + |
| 27 | +### Instrumented Operations |
| 28 | + |
| 29 | +| Span Name | Component | Attributes | |
| 30 | +|---|---|---| |
| 31 | +| `admin GET/POST /v1/...` | HTTP Admin | http.method, http.route, http.status_code | |
| 32 | +| `controlplane.syncer.publish_snapshot` | Controller | scope, gateway/route/backend counts | |
| 33 | +| `controlplane.infrastructure.reconcile` | Infrastructure | managed_gateways, service/endpoint counts | |
| 34 | +| gRPC server spans | otelgrpc | rpc.service, rpc.method | |
| 35 | + |
| 36 | +## Data Plane |
| 37 | + |
| 38 | +Enable tracing in the data plane config: |
| 39 | + |
| 40 | +```yaml |
| 41 | +observability: |
| 42 | + tracing: |
| 43 | + level: "info" |
| 44 | + format: "json" |
| 45 | + open_telemetry: |
| 46 | + enabled: true |
| 47 | + endpoint: "http://otel-collector:4317" |
| 48 | + protocol: "grpc" |
| 49 | + timeout_ms: 3000 |
| 50 | + insecure: true |
| 51 | + sample_ratio: 0.1 |
| 52 | + service_name: "nantian-dataplane" |
| 53 | + service_namespace: "production" |
| 54 | + service_instance_id: "dp-01" |
| 55 | +``` |
| 56 | + |
| 57 | +| Field | Type | Default | Description | |
| 58 | +|---|---|---|---| |
| 59 | +| `enabled` | bool | `false` | Enable OTLP tracing | |
| 60 | +| `endpoint` | string | — | OTLP collector URL | |
| 61 | +| `protocol` | string | `grpc` | OTLP protocol | |
| 62 | +| `sample_ratio` | float | `1.0` | Sampling ratio | |
| 63 | +| `service_name` | string | `nantian-dataplane` | Service name in traces | |
| 64 | + |
| 65 | +### Instrumented Operations |
| 66 | + |
| 67 | +| Span Name | Component | Attributes | |
| 68 | +|---|---|---| |
| 69 | +| Request span (auto) | Proxy | http.method, url.path, client.address, server.address | |
| 70 | +| Request enriched | Proxy | gateway.listener, route.name/namespace/kind, backend | |
| 71 | +| Response enriched | Proxy | http.status_code, retry.attempts, response_flags | |
| 72 | +| `ai.inference` | AI Gateway | ai.model, ai.format, ai.stream, prompt_tokens, completion_tokens | |
| 73 | +| `ai.first_token` | AI Gateway | ai.first_token_ms | |
| 74 | + |
| 75 | +### W3C Trace Context Propagation |
| 76 | + |
| 77 | +- Incoming `traceparent` headers are extracted and continued |
| 78 | +- Outgoing `traceparent` headers are injected to upstream backends |
| 79 | +- gRPC xDS streams propagate trace context (via otelgrpc) |
| 80 | + |
| 81 | +## End-to-End Trace Flow |
| 82 | + |
| 83 | +``` |
| 84 | +Client Request (traceparent) |
| 85 | + → Data Plane Proxy (extract + continue) |
| 86 | + → AI Gateway Filter (child span: ai.inference) |
| 87 | + → Upstream Backend (inject traceparent) |
| 88 | + ← Data Plane Proxy (enrich + end) |
| 89 | +``` |
| 90 | +
|
| 91 | +Control plane reconciliation traces independently: |
| 92 | +``` |
| 93 | +CRD Change Event |
| 94 | + → Controller Reconciler (span: publish_snapshot) |
| 95 | + → Infrastructure Reconciler (span: reconcile) |
| 96 | + → gRPC xDS Push (span: otelgrpc) |
| 97 | +``` |
| 98 | +
|
| 99 | +## Viewing Traces |
| 100 | +
|
| 101 | +Use any OTLP-compatible backend: |
| 102 | +- **Jaeger**: `jaeger-all-in-one` with OTLP gRPC on port 4317 |
| 103 | +- **Grafana Tempo**: OTLP endpoint |
| 104 | +- **OpenTelemetry Collector**: Forward to any backend |
0 commit comments