`MultiplexInstrument` does not fan out to all tracers

[kukushechkin]

MultiplexInstrument can hold multiple Tracers, but only sends traces to one of them — is this expected?

The asymmetry is that inject/extract fan out to every member of the multiplex, but span creation doesn't: InstrumentationSystem.tracer/legacyTracer and the free-function withSpan/startSpan resolve to the first Tracer in the
multiplex and ignore the rest. So context is propagated through all members, but only one of them ever receives spans.

import Tracing
import InMemoryTracing

let tracerA = InMemoryTracer()
let tracerB = InMemoryTracer()
InstrumentationSystem.bootstrap(MultiplexInstrument([tracerA, tracerB]))

withSpan("request") { _ in }

print(tracerA.finishedSpans.count)  // 1
print(tracerB.finishedSpans.count)  // 0  ← tracerB never sees the span

So my questions:

Is "only the first Tracer receives spans" the intended contract? If so, it would be worth documenting on MultiplexInstrument, since "use multiple instruments" reads as though all of them are active. If not, should span creation fan out to every Tracer in the multiplex?

[slashmo]

Good catch! I never really considered the multi-tracer use case. When we created the multiplex instrument we were thinking about the instrument protocol itself, so injecting and extracting service context values, not the refined tracer protocol. That being said, I agree that the current UX is not ideal. The way I see it is we have two options:

1. Emit some kind of warning letting the user know about this limitation at runtime
2. Implement a multi-tracer capable span implementation and MultiplexTracer that acts as a multiplex to multiple concrete span implementations, forwarding span creation, attribute updates, error recording etc.
3. ???

Before properly evaluating the second option I'd love to know if you had a specific use-case for having multiple tracers in parallel.

@ktoso I'd also love to hear your thoughts on this.

[ktoso]

We discussed this offline, imho it's just a naive bug that we had assumed there's one tracer.

In most applications that's absolutely true, you don't really mix tracing systems -- that'd defeat the purpose of connecting all spans into a single hierarchy. But given that we're about to allow scopes I think it's absolutely permissible to emit to multiple ones, although I still don't really see good reasons to do so, there's nothing really we need to ban here.

The current behavior id classify as accident and fix it by emitting to all tracers we have in the multiplex one.

[FranzBusch]

In most applications that's absolutely true, you don't really mix tracing systems -- that'd defeat the purpose of connecting all spans into a single hierarchy. But given that we're about to allow scopes I think it's absolutely permissible to emit to multiple ones, although I still don't really see good reasons to do so, there's nothing really we need to ban here.

After having thought about this more, I think there are clear use-cases if you want to emit the spans into multiple systems at once. This is essentially what Otel allows as well where it allows multiple exporters. Of course, we expect most folks to use Otel and then setup multiple exporters but it is also reasonable to want to do this without Otel.

[kukushechkin]

I agree we should go with a MultiplexTracer and MultiplexSpan which can correctly handle multiple tracers. Then 3 implementation details that I think make it work:

• Context is accumulated through tracers startSpan/startAnySpan returning span.context and MultiplexSpan gets the accumulated via MultiplexSpan.init(context: ServiceContext, spans: [any Tracing.Span]).
• MultplixTracer.init(tracers: [any Tracer], otherInstruments: [any Instrument] = []) takes both tracers and instruments, and makes a MultiplexInstrument out of otherInstruments (which already works correctly).
• MultiplexInstrument gets a warning if multiple tracers.