feat: Add FFI for trace exporter#1952
Open
lloeki wants to merge 14 commits into
Open
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
96342f8 to
303d914
Compare
🎉 All green!❄️ No new flaky tests detected 🎯 Code Coverage (details) 🔗 Commit SHA: 9d7998b | Docs | Datadog PR Page | Give us feedback! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1952 +/- ##
==========================================
+ Coverage 71.79% 72.75% +0.96%
==========================================
Files 434 452 +18
Lines 70598 74562 +3964
==========================================
+ Hits 50687 54249 +3562
- Misses 19911 20313 +402
🚀 New features to boost your workflow:
|
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
303d914 to
2b39efc
Compare
Introduce an opaque `TracerSpan` handle wrapping `Span<BytesData>` and expose it to C callers via four FFI functions: - `ddog_tracer_span_new`: create a span with all scalar fields - `ddog_tracer_span_free`: release the span - `ddog_tracer_span_set_meta`: add a string tag - `ddog_tracer_span_set_metric`: add a numeric tag This enables language tracers (e.g. Ruby) to build spans field-by-field through the C API, bypassing msgpack serialization on the caller side.
Introduce an opaque `TracerTraceChunks` handle wrapping `Vec<Vec<SpanBytes>>` and expose it to C callers via four FFI functions: - `ddog_tracer_trace_chunks_new`: create a container with optional capacity hint - `ddog_tracer_trace_chunks_free`: release the container - `ddog_tracer_trace_chunks_begin_chunk`: start a new trace chunk - `ddog_tracer_trace_chunks_push_span`: move a `TracerSpan` into the current chunk, consuming it Spans are consumed on push to enforce single-ownership and prevent double-use from C callers.
Wire `TracerTraceChunks` to the existing `TraceExporter::send_trace_chunks` method through a new C-callable function. The chunks are consumed on call and the agent response is optionally written to an out-parameter. This completes the span-building FFI surface: callers can now construct spans via `ddog_tracer_span_*`, group them via `ddog_tracer_trace_chunks_*`, and send them via `ddog_trace_exporter_send_trace_chunks` — all without serializing to msgpack on the caller side. The `TraceExporter` type alias in `trace_exporter.rs` is widened to `pub(crate)` to allow cross-module access.
lloeki
force-pushed
the
lloeki/ffi-for-trace-exporter
branch
from
2b39efc to
2701231
Compare
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
yannham
reviewed
May 7, 2026
Replace the `match` with identity `Ok` arm with `map_err`, which is the idiomatic Rust combinator for transforming only the error side of a `Result`.
The inner fields are only accessed within the `tracer` module itself, so `pub(crate)` is unnecessarily broad. Default (private) visibility is sufficient since the module owns all access.
`Vec::with_capacity(0)` and `Vec::new()` are identical — both create an empty vec with no heap allocation — so the branch is unnecessary.
…race_chunks`
Replace the `match` with `let Some(...) = ... else { return ... }`,
which is the idiomatic Rust pattern for early-return on a refutable
binding (stabilized in Rust 1.65).
`make_minimal_span` and `make_chunks` have no safety preconditions for their callers — all unsafety is self-contained. Move the `unsafe` from the function signature into internal blocks so the functions present a safe interface.
Extract the span-building logic into an inner closure that returns `Result`, allowing the `?` operator to propagate conversion errors. The outer body calls `inner().err()` to map `Ok(()) -> None` and `Err(e) -> Some(e)`, matching the FFI return type.
Replace `NonNull<Box<T>>` with `&mut MaybeUninit<Box<T>>` for out-parameters in `ddog_tracer_span_new` and `ddog_tracer_trace_chunks_new`. This makes the uninitialized nature of the slot explicit in the type system and allows using `MaybeUninit::write` instead of raw `ptr::write`. Note that `&mut Box<T>` cannot be used here: assignment through `&mut Box<T>` drops the old value first, which is undefined behavior when the slot is uninitialized. `MaybeUninit::write` overwrites without dropping, which is the correct semantics for out-parameters. cbindgen produces identical C signatures (`T **out_handle`) for all three representations.
hoolioh
requested changes
May 7, 2026
Restore the established `NonNull<Box<T>>` convention used throughout `trace_exporter.rs` for FFI out-parameters. While `&mut MaybeUninit` is technically more precise, it is less idiomatic for FFI and inconsistent with the rest of the module. cbindgen produces identical C headers for both representations.
Replace 11 positional parameters with a `#[repr(C)]` struct passed by reference. This avoids holding all fields on the stack per call and allows adding or changing fields without breaking the function signature. Follows the same pattern as `TelemetryClientConfig` elsewhere in the codebase.
Accept a capacity hint for pre-allocating the inner span vector, avoiding reallocations when the number of spans is known beforehand.
…egin_chunk` Change the return type from void to `Option<Box<ExporterError>>` and wrap the body in `catch_panic!` with a null handle check, making the function consistent with every other mutating function in the tracer FFI API.
Member
Author
|
Review comments addressed. If acceptable, I suggest we merge this one as a first iteration of the FFI API. A new stacked PR based on this one is brewing, which has more contentious API changes coming from the dd-trace-rb side review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add span-building and trace-sending FFI functions to
libdd-data-pipeline-ffi, enabling language tracers to construct spans field-by-field through the C API instead of passing pre-serialized msgpack.New types:
TracerSpan: opaque handle wrappingSpan<BytesData>TracerTraceChunks: opaque handle wrappingVec<Vec<SpanBytes>>New functions:
ddog_tracer_span_new,_free,_set_meta,_set_metricddog_tracer_trace_chunks_new,_free,_begin_chunk,_push_spanddog_trace_exporter_send_trace_chunksMotivation
The Ruby tracer needs a native trace export path that bypasses Ruby-side msgpack serialization. This provides the FFI surface for it; the corresponding C extension in
dd-trace-rbcalls these functions.Continuation of the prototype in #1661, reworked to stay close to
main: the genericTraceExporter<H>andSharedRuntimeare untouched; this is purely additive.Additional Notes
Spans are consumed on push, chunks are consumed on send — single-ownership enforced at the API level.
libdd-trace-utilsis promoted from dev-dependency to regular dependency since the FFI crate now needsSpanBytesat build time.Sibling PR for Ruby is at: DataDog/dd-trace-rb#5690
How to test the change?
cargo test -p libdd-data-pipeline-ffi --lib— 39 tests, all passing. The existingtrace_exportertests are unaffected.