Add support for OpenTelemetry Logs API #6465

mabdinur · 2025-09-18T00:23:20Z

What does this PR do?

This PR adds OpenTelemetry Logs support to dd-trace-js, enabling users to emit structured log data that integrates with the OpenTelemetry ecosystem. The implementation includes:

OpenTelemetry Logs API Integration: Full compatibility with @opentelemetry/api-logs
OTLP Export: Logs exported via OpenTelemetry Protocol (OTLP) over HTTP
Protocol Configuration: Support for OTEL_EXPORTER_OTLP_LOGS_PROTOCOL and OTEL_EXPORTER_OTLP_PROTOCOL to choose between http/protobuf and http/json serialization
Batch Processing: Efficient batch processing with configurable sizes and timeouts
Protobuf and JSON serialization using vendored OTLP v1.7.0 proto definitions

Motivation

OpenTelemetry has become the standard for observability in the cloud-native ecosystem. This implementation enables users to:

Use a single library for both tracing and logging
Export logs to any OTLP-compatible backend
Choose between efficient protobuf or human-readable JSON serialization

Checklist

Additional Notes

github-actions · 2025-09-18T20:11:36Z

Overall package size

Self size: 12.72 MB
Deduped: 113.45 MB
No deduping: 128.47 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.2.1 | 20.64 MB | 20.65 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.10.0 | 9.91 MB | 10.3 MB | | @opentelemetry/resources | 1.9.1 | 306.54 kB | 7.46 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.4 | 2.95 MB | 5.73 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api-logs | 0.205.0 | 201.51 kB | 1.42 MB | | @opentelemetry/api | 1.9.0 | 1.22 MB | 1.22 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.4 | 123.18 kB | 851.76 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | source-map | 0.7.6 | 185.63 kB | 185.63 kB | | pprof-format | 2.2.1 | 163.06 kB | 163.06 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

codecov · 2025-09-18T20:11:57Z

Codecov Report

❌ Patch coverage is 94.09722% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.00%. Comparing base (b6077e7) to head (7f4fbff).
⚠️ Report is 17 commits behind head on master.

Files with missing lines	Patch %	Lines
...e/src/opentelemetry/logs/otlp_http_log_exporter.js	84.12%	10 Missing ⚠️
...d-trace/src/opentelemetry/logs/otlp_transformer.js	94.84%	5 Missing ⚠️
...race/src/opentelemetry/logs/batch_log_processor.js	91.30%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6465      +/-   ##
==========================================
+ Coverage   83.97%   84.00%   +0.02%     
==========================================
  Files         489      497       +8     
  Lines       20471    20940     +469     
==========================================
+ Hits        17191    17591     +400     
- Misses       3280     3349      +69

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2025-09-18T20:20:59Z

Benchmarks

Benchmark execution time: 2025-10-10 19:49:53

Comparing candidate commit 7f4fbff in PR branch munir/otlp-logs-support with baseline commit b6077e7 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1600 metrics, 70 unstable metrics.

packages/dd-trace/src/opentelemetry/logs/batch_log_processor.js

packages/dd-trace/src/opentelemetry/logs/logger.js

packages/dd-trace/src/config_defaults.js

datadog-official · 2025-09-19T20:54:23Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 7f4fbff | Docs | Was this helpful? Give us feedback!}

…rsion

… params

…ter and improve tests

khanayan123

Overall LGTM, just a few small nits. Could we also add an integration test under ./integration-test to cover the client flow?

packages/dd-trace/src/config.js

khanayan123 · 2025-10-07T17:17:42Z

packages/dd-trace/src/config.js

+
+    // Disable log injection when OTEL logs are enabled
+    // OTEL logs and DD log injection are mutually exclusive
+    if (this._env.otelLogsEnabled) {


Could we add a unit test for this scenario?

packages/dd-trace/src/config.js

khanayan123 · 2025-10-07T19:04:06Z

packages/dd-trace/src/proxy.js

    }
  }

+  #initializeOpenTelemetryLogs (config) {


NIT: could we do the initialization in ./opentelemetry/logs/index.js ?

mabdinur · 2025-10-08T21:03:36Z

otelLogsEnabled

I am validating this behavior in the system test so I think it's unnessary but I can add it if it's a hard requirement to merge this PR: DataDog/system-tests#5437

BridgeAR

LGTM, while I would prefer to have a few more tests for handling all different data types.
The http part is mocked, so I understand that this is not yet tested. I just wondered, if we want to mock the http layer. That way that would also be tested and we can rely on even more parts in the tests.

* first attempt at otel logs support * lint files and centralize configs * add tests * clean up yarn.lock * clean up logs * add back log exporter logic * clean up docs * clean up docs and fix otlp protocol * revert readme docs * add new otel files * working version * fmy * fmt * Update packages/dd-trace/src/config_defaults.js * fix tests * allow any version of logs api, let opentelemetry api determine the version * add otlp payload tests * add telemetry metrics * some other clean ups * simplify tests * use agent hostname to resolve otlp endpoints * clean up initalization * parse additional otlp headers * clean up component args * clean up docs * clean up component args * remove addLogProcessor, init provider with a processor * support trace-log correlation * clean up registering provider in tests, and rename exporter arg * first round of clean ups from PR review * clean ups part 2 * make things private and clean up tests * clean up tests * clean up yarn file * remove unused configs * fix context issues * nother round of clean ups * group payloads by instrumentation scope * fix typing * address review comments * add better typing, and better support for sending schemaurl * revert instrumentationScope change to span * review comments * clean up protobuf loader file * lint * move protos to same dir, this will set up metrics work * clean up throws * update protos * disable log injection when otel logs support is enabled * update configurations to pass telemetry system tests * remove useless import * add more tests * provide fix for failing system test * add test case for noop logger and fix mocking for remote config * add test case for noop logger and fix mocking for remote config * fix encoding for doubles, remove unused shutdown code, test getLogger params * remove unused timer and shutdown logic, simplify loggerprovider register and improve tests * fix comment * clean up how otel endpoint configs are loaded * address Ayans comments * add integration tests

mabdinur added 4 commits September 17, 2025 19:50

first attempt at otel logs support

343c772

lint files and centralize configs

b3a2973

add tests

2c0d528

clean up yarn.lock

6c359ce

mabdinur added 6 commits September 19, 2025 10:53

clean up logs

7dc1480

add back log exporter logic

2c60dfc

clean up docs

dabe98f

clean up docs and fix otlp protocol

b7ae33d

revert readme docs

f73d425

add new otel files

543805a

mabdinur force-pushed the munir/otlp-logs-support branch from 3de2a5c to 543805a Compare September 19, 2025 18:38

working version

c63e20e

mabdinur commented Sep 19, 2025

View reviewed changes

packages/dd-trace/src/opentelemetry/logs/batch_log_processor.js Show resolved Hide resolved

mabdinur commented Sep 19, 2025

View reviewed changes

packages/dd-trace/src/opentelemetry/logs/logger.js Outdated Show resolved Hide resolved

mabdinur and others added 2 commits September 19, 2025 15:07

fmy

d4d8aac

fmt

9514e68

mabdinur commented Sep 19, 2025

View reviewed changes

packages/dd-trace/src/config_defaults.js Outdated Show resolved Hide resolved

Update packages/dd-trace/src/config_defaults.js

df7cbca

mabdinur force-pushed the munir/otlp-logs-support branch from c5f752b to df7cbca Compare September 19, 2025 20:48

mabdinur force-pushed the munir/otlp-logs-support branch from 57ab573 to 9eded3b Compare September 19, 2025 20:56

fix tests

2c0a5a5

mabdinur force-pushed the munir/otlp-logs-support branch from 9eded3b to 2c0a5a5 Compare September 19, 2025 20:57

mabdinur added 3 commits September 19, 2025 17:08

allow any version of logs api, let opentelemetry api determine the ve…

f109cf3

…rsion

add otlp payload tests

0c29478

add telemetry metrics

8b1ce46

mabdinur changed the title ~~[WIP] Add support for OpenTelemetry Logs API~~ Add support for OpenTelemetry Logs API Sep 22, 2025

add test case for noop logger and fix mocking for remote config

266a239

mabdinur force-pushed the munir/otlp-logs-support branch 3 times, most recently from a520a22 to a890b1f Compare October 6, 2025 21:40

add test case for noop logger and fix mocking for remote config

df5d6b4

mabdinur force-pushed the munir/otlp-logs-support branch from a890b1f to df5d6b4 Compare October 6, 2025 21:40

mabdinur and others added 5 commits October 6, 2025 17:51

Merge branch 'master' into munir/otlp-logs-support

2763296

fix encoding for doubles, remove unused shutdown code, test getLogger…

e995e0c

… params

remove unused timer and shutdown logic, simplify loggerprovider regis…

e20668e

…ter and improve tests

fix comment

3f8eab1

clean up how otel endpoint configs are loaded

148d47b

khanayan123 reviewed Oct 7, 2025

View reviewed changes

address Ayans comments

9ac86bb

mabdinur requested a review from khanayan123 October 8, 2025 21:20

BridgeAR previously approved these changes Oct 9, 2025

View reviewed changes

khanayan123 previously approved these changes Oct 9, 2025

View reviewed changes

mabdinur dismissed stale reviews from khanayan123 and BridgeAR via a4df6c1 October 9, 2025 20:24

mabdinur force-pushed the munir/otlp-logs-support branch from a4df6c1 to cea4a8a Compare October 10, 2025 16:41

add integration tests

7f4fbff

mabdinur force-pushed the munir/otlp-logs-support branch from cea4a8a to 7f4fbff Compare October 10, 2025 19:37

khanayan123 approved these changes Oct 10, 2025

View reviewed changes

mabdinur requested a review from BridgeAR October 10, 2025 20:23

BridgeAR approved these changes Oct 12, 2025

View reviewed changes

mabdinur merged commit 2555791 into master Oct 14, 2025
742 of 746 checks passed

mabdinur deleted the munir/otlp-logs-support branch October 14, 2025 15:40

dd-octo-sts bot mentioned this pull request Oct 15, 2025

v5.72.0 proposal #6666

Merged

Add support for OpenTelemetry Logs API #6465

Add support for OpenTelemetry Logs API #6465

Uh oh!

Conversation

mabdinur commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Checklist

Additional Notes

Uh oh!

github-actions bot commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overall package size

Uh oh!

codecov bot commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter bot commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Uh oh!

Uh oh!

Uh oh!

Uh oh!

datadog-official bot commented Sep 19, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

khanayan123 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

khanayan123 Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

khanayan123 Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

mabdinur commented Oct 8, 2025

Uh oh!

BridgeAR left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mabdinur commented Sep 18, 2025 •

edited

Loading

github-actions bot commented Sep 18, 2025 •

edited

Loading

codecov bot commented Sep 18, 2025 •

edited

Loading

pr-commenter bot commented Sep 18, 2025 •

edited

Loading

datadog-official bot commented Sep 19, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading