perf(native-spans): bound exporter batching - #9680
Closed
BridgeAR wants to merge 6 commits into
Closed
Conversation
## Summary Keep the existing flush timer authoritative after an in-flight native send and trigger an early flush at 2,000 pending spans. ## Why The native exporter waited for the two-second interval before its first send, then bypassed batching after every settlement. On the Express/PostgreSQL workload, bounded batching reduced CPU/request by 18.6%, increased throughput by 25.1%, and reduced RSS from 1,344 MiB to 366 MiB. ## Test plan - Run the native exporter unit tests. - Verify every changed production branch with c8. - Run the full repository lint. - Run three fresh-process 50,000-request Express/PostgreSQL trials with exact trace and query counts.
Contributor
Overall package sizeSelf size: 7.7 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.3 | 125.43 kB | 441.68 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## bengl/native-spans-attempt-3 #9680 +/- ##
===============================================================
Coverage ? 98.49%
===============================================================
Files ? 953
Lines ? 134352
Branches ? 11416
===============================================================
Hits ? 132328
Misses ? 2024
Partials ? 0 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
All spans in a trace share an immutable ID, but the native path rebuilt its 16-byte representation for every child. Reusing the shared representation reduced buildNativeTraceId self-time from 94.6 to 30.8 ms in a 50,000-request Express/PostgreSQL profile and its isolated seven-span path from 1,081 to 169 ns/trace.
Assigning and deleting the construction-time name hook put every native span context on a slow object shape even though final synchronization already owns name writes. Removing the stale hook reduced CPU per request from 112.41 to 99.77 µs and raised throughput from 11,039 to 12,175 requests/s in the Express/PostgreSQL workload; isolated construction fell from 75.1 to 6.4 ns/span.
## Summary Keep resource names intact in native fast final sync, matching the JS v0.4 and v0.5 encoders. ## Why The tracer applied the agent's 5,000-character normalization limit before native export. Long SQL resources then differed between exporters and failed the existing PostgreSQL wire assertions. ## Test plan - ./node_modules/.bin/mocha packages/dd-trace/test/native/span_context.spec.js - Existing PostgreSQL long-query cases for pg 8.0.3 and 8.22 (2 passing)
## Summary Mirror the canonical formatter's base-service inference in native fast final sync and remove the now-redundant configured-service field from native contexts. ## Why The fast path registered an overridden service but skipped _dd.base_service. Raw spans and WASM output therefore lost the configured service whenever a span selected a different one. ## Test plan - Native tracer and span-context suites (45 passing) - Native plugin wire assertions for base-service propagation (3 passing)
BenchmarksBenchmark execution time: 2026-08-04 19:45:34 Comparing candidate commit 78e21e3 in PR branch Found 27 performance improvements and 1 performance regressions! Performance is the same for 2177 metrics, 99 unstable metrics.
|
## Summary Change-queue flushing consumed 95.8% of the deferred-finish profile and kept the eight-sample CI variant running when the 30-minute job expired. ## Why The benchmark is meant to isolate span construction and finish, but it applied and exported every queued native mutation. Discarding those mutations reduced the same 250,000-span process from 23.11 s to 0.60 s. Native event samples still drain because libdatadog applies events directly. ## Test plan - Run all span variants through three fresh sirun matrices. - Run changed-line coverage and full lint.
bengl
force-pushed
the
BridgeAR/2026-08-04-bound-native-export-batches
branch
2 times, most recently
from
August 6, 2026 18:19
cbf2cbf to
78e21e3
Compare
Member
Author
|
These changes are already incoporated |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep the existing flush timer authoritative after an in-flight native send and flush early at 2,000 pending spans. This draft is stacked on #9139.
Why
The stacked base waits up to two seconds before its first send, then bypasses batching after every settlement. Preserving the timer without a bound retained 1,269–1,773 MiB RSS. Libdatadog does not expose the pre-serialization payload size, so this uses a span-count bound.
The 2,000-span cap was the measured tradeoff. Raising it to 4,000 gained 0.4% CPU while adding 8.9% RSS. Raising it to 8,000 regressed CPU and averaged 633 MiB RSS.
Across three rotated Express/PostgreSQL trials with 50,000 requests each:
Every trial delivered 50,000 traces and executed 50,000 PostgreSQL queries.
Test plan