Skip to content

[BUG]: LLMObs injection duplicates _dd.p.llmobs_ml_app in x-datadog-tags #9714

Description

@llewekam

Tracer Version(s)

5.118.0

Node.js Version(s)

24.18.0

Bug Report

With LLM Observability enabled, the LLMObs inject hook appends _dd.p.llmobs_ml_app to x-datadog-tags without checking whether the standard tag propagator has already written it. Any trace that already carries the tag emits it twice on the next outbound request.

Output from the reproduction script:

inbound  ml_app x1 : _dd.p.tid=6a734d1a00000000,_dd.p.dm=-0,_dd.p.llmobs_ml_app=my-ml-app
outbound ml_app x2 : _dd.p.tid=6a734d1a00000000,_dd.p.dm=-0,_dd.p.llmobs_ml_app=my-ml-app,_dd.p.llmobs_ml_app=my-ml-app

Reproduction Code

'use strict';
process.env.DD_LLMOBS_ENABLED = '1';
process.env.DD_LLMOBS_ML_APP = 'my-ml-app';
process.env.DD_LLMOBS_AGENTLESS_ENABLED = '1';
process.env.DD_API_KEY = 'dummy-key-for-repro';
process.env.DD_TRACE_STARTUP_LOGS = 'false';

const tracer = require('dd-trace').init();
const count = (c) =>
((c['x-datadog-tags'] || '').match(/_dd.p.llmobs_ml_app=/g) || []).length;

// An inbound request that already carries the ml_app tag (as written by an
// upstream service), then this service makes its own outbound request.
const inbound = {
'x-datadog-trace-id': '1234567890',
'x-datadog-parent-id': '1234567890',
'x-datadog-sampling-priority': '1',
'x-datadog-tags': '_dd.p.tid=6a734d1a00000000,_dd.p.dm=-0,_dd.p.llmobs_ml_app=my-ml-app',
};

const span = tracer.startSpan('outbound', { childOf: tracer.extract('http_headers', inbound) });
const outbound = {};
tracer.inject(span, 'http_headers', outbound);
span.finish();

console.log('inbound ml_app x' + count(inbound), ':', inbound['x-datadog-tags']);
console.log('outbound ml_app x' + count(outbound), ':', outbound['x-datadog-tags']);

Error Logs

No response

Tracer Config

No response

Operating System

No response

Bundling

Unsure

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions