Skip to content

Commit 5876427

Browse files
lloekiekump
andauthored
feat(data-pipeline): set structured span values (#2300)
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Adds direct structured-value insertion into `TracerSpan.meta_struct`, reusing the validator and encoder from [DataDog/libdatadog PR 2304](#2304) while moving the encoded allocation directly into the span. The opaque blob and standalone encoder APIs remain available. # Motivation The standalone encoder still requires consumers to manage an intermediate encoded blob and an additional value copy. The direct setter provides the final efficient API for structured payload parity tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # Additional Notes **Stack position: L5, current PR: [DataDog/libdatadog PR 2300](#2300), stacked on L4 [DataDog/libdatadog PR 2304](#2304 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | L0 | [DataDog/libdatadog PR 2302](#2302) | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) | | L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) | | L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) | | **L5 (current)** | **[DataDog/libdatadog PR 2300](#2300 | **[R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133 | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. This PR remains a draft because its review and final rebase depend on L4 and, transitively, L1. Its matching consumer [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) is also a draft pending this API, the v39 package, and [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). L4 remains draft until L1 is ready. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop --command cargo test -p libdd-data-pipeline-ffi --locked` passed 68 tests, including direct-setter/standalone-encoder equivalence, atomic failure, invalid handles, overwrite behaviour, and ownership. - `nix develop --command cargo clippy -p libdd-data-pipeline-ffi --all-targets --locked -- -D warnings` passed. - `nix develop .#nightly --command cargo fmt --all -- --check` passed. - The release artifact built successfully with `nix develop -c cargo run --bin release -- --out /tmp/opencode/libdatadog-m3/x86_64-linux`. - The matching consumer's native suite passed 122 examples, and AppSec passed 807 examples against the local artifact. - `nix develop .#ruby40 --command nix shell nixpkgs#valgrind --command bundle exec ruby_memcheck vendor/bundle/ruby/4.0.0/gems/rspec-core-3.13.6/exe/rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/send_traces_spec.rb spec/datadog/tracing/transport/native/trace_exporter_spec.rb` passed with no Valgrind memory errors. - Full Standard and Steep checks passed on the matching consumer branch. [APMSP-3219]: https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ekump <edmund.kump@datadoghq.com>
1 parent f3d3d80 commit 5876427

2 files changed

Lines changed: 150 additions & 12 deletions

File tree

libdd-data-pipeline-ffi/src/structured_value.rs

Lines changed: 19 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,24 @@ fn encode_one(
158158
Ok(())
159159
}
160160

161+
pub(crate) fn encode_value(
162+
tokens: Slice<TracerValueToken<'_>>,
163+
) -> Result<Vec<u8>, Box<ExporterError>> {
164+
let tokens = tokens
165+
.try_as_slice()
166+
.map_err(|_| invalid_argument("structured value token slice is invalid"))?;
167+
if tokens.is_empty() {
168+
return Err(invalid_input("structured value token slice is empty"));
169+
}
170+
let mut output = Vec::new();
171+
let mut index = 0;
172+
encode_one(tokens, &mut index, 0, &mut output)?;
173+
if index != tokens.len() {
174+
return Err(invalid_input("structured value has trailing tokens"));
175+
}
176+
Ok(output)
177+
}
178+
161179
/// Encode one flat preorder structured value as an owned MessagePack blob.
162180
///
163181
/// On success, `out_handle` receives an owned blob that must be freed with
@@ -178,18 +196,7 @@ pub unsafe extern "C" fn ddog_tracer_encode_value(
178196
catch_panic!(
179197
{
180198
let inner = || -> Result<(), Box<ExporterError>> {
181-
let tokens = tokens
182-
.try_as_slice()
183-
.map_err(|_| invalid_argument("structured value token slice is invalid"))?;
184-
if tokens.is_empty() {
185-
return Err(invalid_input("structured value token slice is empty"));
186-
}
187-
let mut output = Vec::new();
188-
let mut index = 0;
189-
encode_one(tokens, &mut index, 0, &mut output)?;
190-
if index != tokens.len() {
191-
return Err(invalid_input("structured value has trailing tokens"));
192-
}
199+
let output = encode_value(tokens)?;
193200
out_handle
194201
.as_ptr()
195202
.write(Box::new(TracerEncodedValue(output)));

libdd-data-pipeline-ffi/src/tracer.rs

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
1212
use crate::error::{ExporterError, ExporterErrorCode as ErrorCode};
1313
use crate::response::ExporterResponse;
14+
use crate::structured_value::{encode_value, TracerValueToken};
1415
use crate::trace_exporter::TraceExporter;
1516
use crate::{catch_panic, gen_error};
1617
use libdd_common_ffi::slice::{AsBytes, ByteSlice, Slice};
@@ -263,6 +264,43 @@ pub unsafe extern "C" fn ddog_tracer_span_set_meta_struct_blob(
263264
)
264265
}
265266

267+
/// Encode and add or overwrite a structured metadata entry (`meta_struct`) on
268+
/// the span.
269+
///
270+
/// The `key` is copied into the span. The structured-value tokens are fully
271+
/// validated and encoded directly into the span without exposing an
272+
/// intermediate blob to the caller. The span is unchanged on failure.
273+
///
274+
/// # Safety
275+
///
276+
/// `handle` must be a valid pointer to a `TracerSpan`. `key` must point to
277+
/// valid UTF-8 memory. `tokens` and every byte slice referenced by its tokens
278+
/// must remain valid for this call.
279+
#[no_mangle]
280+
pub unsafe extern "C" fn ddog_tracer_span_set_meta_struct(
281+
handle: Option<&mut TracerSpan>,
282+
key: CharSlice,
283+
tokens: Slice<TracerValueToken<'_>>,
284+
) -> Option<Box<ExporterError>> {
285+
catch_panic!(
286+
if let Some(span) = handle {
287+
let key = match charslice_to_bytesstring(key) {
288+
Ok(key) => key,
289+
Err(error) => return error,
290+
};
291+
let value = match encode_value(tokens) {
292+
Ok(value) => value,
293+
Err(error) => return Some(error),
294+
};
295+
span.0.meta_struct.insert(key, Bytes::from(value));
296+
None
297+
} else {
298+
gen_error!(ErrorCode::InvalidArgument)
299+
},
300+
gen_error!(ErrorCode::Panic)
301+
)
302+
}
303+
266304
/// Replace all span links in one atomic operation.
267305
///
268306
/// The links, attributes, and strings are copied before this function returns.
@@ -755,6 +793,10 @@ pub unsafe extern "C" fn ddog_trace_exporter_send_trace_chunks(
755793
mod tests {
756794
use super::*;
757795
use crate::error::ddog_trace_exporter_error_free;
796+
use crate::structured_value::{
797+
ddog_tracer_encode_value, ddog_tracer_encoded_value_as_slice,
798+
ddog_tracer_encoded_value_free, TracerEncodedValue,
799+
};
758800
use std::mem::MaybeUninit;
759801

760802
fn cs(s: &str) -> CharSlice<'_> {
@@ -765,6 +807,18 @@ mod tests {
765807
ByteSlice::from(bytes)
766808
}
767809

810+
fn nil_token() -> TracerValueToken<'static> {
811+
TracerValueToken {
812+
kind: 0,
813+
bool_value: 0,
814+
child_count: 0,
815+
i64_value: 0,
816+
u64_value: 0,
817+
f64_value: 0.0,
818+
bytes: ByteSlice::empty(),
819+
}
820+
}
821+
768822
fn make_minimal_span() -> Box<TracerSpan> {
769823
unsafe {
770824
let mut handle = MaybeUninit::<Box<TracerSpan>>::uninit();
@@ -1211,6 +1265,83 @@ mod tests {
12111265
}
12121266
}
12131267

1268+
#[test]
1269+
fn set_meta_struct_matches_standalone_encoder_without_copying_value() {
1270+
unsafe {
1271+
let tokens = [nil_token()];
1272+
let token_slice = Slice::from(tokens.as_slice());
1273+
let expected = {
1274+
let mut handle = MaybeUninit::<Box<TracerEncodedValue>>::uninit();
1275+
let out = NonNull::new(handle.as_mut_ptr()).unwrap();
1276+
assert!(ddog_tracer_encode_value(token_slice, out).is_none());
1277+
let blob = handle.assume_init();
1278+
let bytes = ddog_tracer_encoded_value_as_slice(Some(&blob))
1279+
.as_bytes()
1280+
.to_vec();
1281+
ddog_tracer_encoded_value_free(Some(blob));
1282+
bytes
1283+
};
1284+
1285+
let mut span = make_minimal_span();
1286+
let encoded = encode_value(token_slice).unwrap();
1287+
let encoded_ptr = encoded.as_ptr();
1288+
let bytes = Bytes::from(encoded);
1289+
assert_eq!(bytes.as_ptr(), encoded_ptr);
1290+
1291+
assert!(
1292+
ddog_tracer_span_set_meta_struct(Some(&mut *span), cs("key"), token_slice,)
1293+
.is_none()
1294+
);
1295+
assert_eq!(span.0.meta_struct.get("key").unwrap().as_ref(), expected);
1296+
1297+
ddog_tracer_span_free(Some(span));
1298+
}
1299+
}
1300+
1301+
#[test]
1302+
fn set_meta_struct_failure_does_not_replace_existing_value() {
1303+
unsafe {
1304+
let mut span = make_minimal_span();
1305+
assert!(ddog_tracer_span_set_meta_struct_blob(
1306+
Some(&mut *span),
1307+
cs("key"),
1308+
bs(b"existing"),
1309+
)
1310+
.is_none());
1311+
1312+
let empty: &[TracerValueToken<'_>] = &[];
1313+
let error =
1314+
ddog_tracer_span_set_meta_struct(Some(&mut *span), cs("key"), Slice::from(empty));
1315+
assert!(error.is_some());
1316+
assert_eq!(span.0.meta_struct.get("key").unwrap().as_ref(), b"existing");
1317+
ddog_trace_exporter_error_free(error);
1318+
1319+
let invalid_key = CharSlice::from_bytes(&[0xff]);
1320+
let tokens = [nil_token()];
1321+
let error = ddog_tracer_span_set_meta_struct(
1322+
Some(&mut *span),
1323+
invalid_key,
1324+
Slice::from(tokens.as_slice()),
1325+
);
1326+
assert!(error.is_some());
1327+
assert_eq!(span.0.meta_struct.get("key").unwrap().as_ref(), b"existing");
1328+
ddog_trace_exporter_error_free(error);
1329+
1330+
ddog_tracer_span_free(Some(span));
1331+
}
1332+
}
1333+
1334+
#[test]
1335+
fn set_meta_struct_null_handle_returns_error() {
1336+
unsafe {
1337+
let tokens = [nil_token()];
1338+
let error =
1339+
ddog_tracer_span_set_meta_struct(None, cs("key"), Slice::from(tokens.as_slice()));
1340+
assert!(error.is_some());
1341+
ddog_trace_exporter_error_free(error);
1342+
}
1343+
}
1344+
12141345
#[test]
12151346
fn set_meta_null_handle_returns_error() {
12161347
unsafe {

0 commit comments

Comments
 (0)