feat: add per-request correlation ID to logs

[shanduur]

Generate (or honor inbound X-Request-ID) a request ID per HTTP
request, bind a request-scoped logger carrying it in the context,
and echo the ID in the response header. All logs for one request
now share a request_id field.

Threaded through the build path: the request logger is carried
into detached/singleflight build contexts (installer image and
asset builder) and into schematic Put, so build and create logs
keep the originating request_id even after the request returns.

Artifacts fetch helpers (behind singleflight, shared across
requests) and background htpasswd watcher logs are intentionally
left on the construction-time logger.

Signed-off-by: Mateusz Urbanek mateusz.urbanek@siderolabs.com

[shanduur]

Just FYI, why I am not using OTel instrumentation (yet).

The first step for better observability is just a per-request correlation ID in logs (request_id, honored from an inbound X-Request-ID header or generated). That's the cheapest high-value increment: zero infra - no collector, no exporter, no backend. We can just query logs with request_id=X and follow a request end-to-end, including across detached/singleflight builds and background work that outlives the request.

Full OTel tracing can be the next step, as it needs a TracerProvider, span instrumentation, a collector (Jaeger/Tempo/OTLP), and sampling decisions. More moving parts and an ops dependency.

And a final note - the two aren't competing, and I've kept the design somewhat OTel-compatible. All request-scoped logging goes through a single seam (ctxlog.Logger(ctx, base)), and honoring an inbound request-id header already mirrors W3C traceparent propagation. When we adopt OTel, trace_id/span_id get injected in that one function and every existing log line picks them up - no call-site changes.

(Also evaluated the otelzap wrapper - skipping it: it forces a logger-type migration across the codebase to duplicate what our seam already centralizes, and adds nothing until spans actually exist.)

[smira]

I would rather skip OTel until we really need it, it's so much heavy on its own. Might be nice though for local development/debugging to understand the request flow.

[shanduur]

/m