Including trace ID in log messages

[blindmonkey]

Hi team!

We're currently using RUM, the network interceptor, and logging, and are trying to make these play nice together. The use case that we're trying to figure out right now is that we use the logger to notify of network request failures, which may include a failure to parse the response. When such an error occurs, it's properly interleaved in the RUM view and that's super helpful. However, the log message doesn't get associated with the network trace, and it's not clear how to make this happen.

I've also tried creating a span and using span.log, but that just doesn't show anywhere -- not in RUM, and not in APM, and certainly not in the trace view for the relevant request. There doesn't really seem to be any good way to get a reference to either the span for the actual network request, or the trace ID; so my current plan is to propagate the URLRequest to our outer network layer, extract the trace ID from the headers, send it as a custom attribute, and configure Datadog to use that as per this documentation I found.

However, this seems like a fairly common use case, so I find myself wondering.. are we missing something obvious?

[maxep]

Hi @blindmonkey 👋

This has never been requested tho it's a valid use case. We currently don't support correlation between distributed tracing and logs on the SDK, and I'm afraid that manually injecting the dd.trace_id as a log attribute won't work: It is considered as a reserved attribute and it will be filtered out during sanitisation.

With the current state of the SDK I don't see any workaround to offer sadly, I will bring this topic to the team as a feature request.

[blindmonkey]

@maxep that's surprising, and I guess I'm confused about why manually specifying the trace ID in our logs won't work. The documentation I linked to earlier appears to imply that we can specify a custom attribute in our logs and configure Datadog to interpret it as a trace ID, which would connect the logs to the trace. Am I misunderstanding the documentation?

[maxep]

Hi @blindmonkey !

Sorry for the confusion, Yes you are right: You can manually inject the Trace ID as a Log custom attribute and configure Trace Id attributes section of your Log preprocessing. You should then be able to see your logs associated with that Trace. But don't use dd.trace_id as your custom attribute name as it is reserved.

We will explore ways to provide the Trace ID automatically when using Distributed Tracing, so I will keep this ticket open 👍

[sschizas]

Any update on this one?

[mariedm]

Hi @sschizas, this hasn't been addressed yet. I'm creating a ticket so that our team can look into it.

[servercimen]

@mariedm @maxep any update on this?

[maxep]

Hey @servercimen 👋
No update to share, you can still use a Log preprocessor as described here to specify which custom attribute to use as trace/span ids.
I hope this helps!

[khsora34]

@blindmonkey @maxep , I was gonna try adding the trace id as a custom attribute, but I dunno how to get the trace id generated for the request.

I see that in the SDK you guys change the request after it has been sent. Do you know a way to get this trace after added by the SDK? I wonder if we could leverage the delegate somehow, but the methods that I've tried either get executed before the injection or they don't have information on the new injected headers.

[simaoseica-dd]

Hi @khsora34
As @maxep mentioned, the SDK currently does not expose an interface to access or share the Trace ID when using Distributed Tracing.

The only way to gain more control over network tracing is by manually creating a custom span. You can find more details in the point 8 of the Tracing iOS Applications documentation. It explains how to access the span context, retrieve the Trace ID, and manually propagate it through your request.