Skip to content

test(ffe): use canonical fixtures#1979

Open
leoromanovsky wants to merge 7 commits into
mainfrom
leo.romanovsky/ffe-canonical-fixtures-20260512
Open

test(ffe): use canonical fixtures#1979
leoromanovsky wants to merge 7 commits into
mainfrom
leo.romanovsky/ffe-canonical-fixtures-20260512

Conversation

@leoromanovsky
Copy link
Copy Markdown
Contributor

@leoromanovsky leoromanovsky commented May 12, 2026

Motivation

Use DataDog/ffe-system-test-data as libdatadog's canonical FFE/OpenFeature JSON fixture source instead of keeping local copied fixtures.

Changes and Decisions

  • Add datadog-ffe/ffe-system-test-data as a submodule pinned to merged canonical main commit 4446371bc1ca52bd526356927ef42d380145b118; remove copied datadog-ffe/tests/data fixtures.
  • Point datadog-ffe tests, benches, the UFC parse test, and the FFI harness at canonical ufc-config.json and evaluation-cases/*.json, with paths anchored from CARGO_MANIFEST_DIR.
  • Assert canonical fixture values only. result.reason stays out of scope while reason behavior is still evolving.
  • Initialize submodules in FFI and benchmark CI checkouts so the fixture files exist where those jobs run.
  • Add CODEOWNERS coverage for .gitmodules and weekly Dependabot gitsubmodule updates.

https://datadoghq.atlassian.net/wiki/spaces/PANA/pages/6713639211/FFE+SDK+Fixture+Contribution+Guide

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 12, 2026

📚 Documentation Check Results

⚠️ 367 documentation warning(s) found

📦 datadog-ffe - 197 warning(s)

📦 tools - 170 warning(s)


Updated: 2026-05-13 15:47:13 UTC | Commit: 16a59da | missing-docs job results

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 12, 2026

🔒 Cargo Deny Results

⚠️ 2 issue(s) found, showing only errors (advisories, bans, sources)

📦 datadog-ffe - 1 error(s)

Show output
error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:116:1
    │
116 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      └── (dev) libdd-common v4.0.0
          └── datadog-ffe v1.0.0

advisories FAILED, bans ok, sources ok

📦 tools - 1 error(s)

Show output
error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:76:1
   │
76 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
   │
   ├ ID: RUSTSEC-2026-0097
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
   ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
     
     - The `log` and `thread_rng` features are enabled
     - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
     - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
     - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
     - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
     
     `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
   ├ Announcement: https://github.com/rust-random/rand/pull/1763
   ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
   ├ rand v0.8.5
     └── (dev) libdd-common v4.0.0
         └── tools v33.0.0

advisories FAILED, bans ok, sources ok

Updated: 2026-05-13 15:49:04 UTC | Commit: 16a59da | dependency-check job results

@github-actions
Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/leo.romanovsky/ffe-canonical-fixtures-20260512

Summary by Rule

Rule Base Branch PR Branch Change

Annotation Counts by File

File Base Branch PR Branch Change

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 6 6 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 3 3 No change (0%)
datadog-sidecar 57 57 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 8 8 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 15 15 No change (0%)
Total 203 203 No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@datadog-prod-us1-3
Copy link
Copy Markdown

datadog-prod-us1-3 Bot commented May 12, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 72.70% (+0.11%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4aefcc6 | Docs | Datadog PR Page | Give us feedback!

@leoromanovsky leoromanovsky changed the title Use canonical FFE fixtures test(ffe): use canonical fixtures May 13, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.69%. Comparing base (866d7c8) to head (4aefcc6).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1979      +/-   ##
==========================================
+ Coverage   72.58%   72.69%   +0.11%     
==========================================
  Files         451      451              
  Lines       74095    74258     +163     
==========================================
+ Hits        53782    53983     +201     
+ Misses      20313    20275      -38     
Components Coverage Δ
libdd-crashtracker 65.31% <ø> (ø)
libdd-crashtracker-ffi 37.68% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 85.97% <ø> (ø)
libdd-data-pipeline-ffi 71.04% <ø> (ø)
libdd-common 79.81% <ø> (ø)
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.34% <ø> (+3.45%) ⬆️
libdd-telemetry-ffi 31.36% <ø> (+11.99%) ⬆️
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 74.75% <ø> (-1.47%) ⬇️
libdd-profiling 81.57% <ø> (ø)
libdd-profiling-ffi 64.51% <ø> (ø)
libdd-sampling 97.25% <ø> (ø)
datadog-sidecar 29.09% <ø> (-0.05%) ⬇️
datdog-sidecar-ffi 9.67% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.39% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 88.85% <ø> (ø)
libdd-tracer-flare 86.88% <ø> (ø)
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 13, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 81.84 MB 81.84 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.57 MB 7.57 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.01 MB 10.01 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 98.03 MB 98.03 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.48 MB 24.48 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 79.87 KB 79.87 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 180.22 MB 180.20 MB -0% (-16.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 913.92 MB 913.92 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 7.73 MB 7.73 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 79.87 KB 79.87 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.17 MB 23.17 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 45.36 MB 45.36 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.09 MB 21.09 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 81.11 KB 81.11 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 184.40 MB 184.40 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 900.39 MB 900.39 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 5.99 MB 5.99 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 81.11 KB 81.11 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 24.80 MB 24.80 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 42.86 MB 42.86 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 72.93 MB 72.93 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.42 MB 8.42 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 90.69 MB 90.69 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.06 MB 10.06 MB 0% (0 B) 👌

Comment thread datadog-ffe/src/rules_based/eval/eval_assignment.rs Outdated
@leoromanovsky leoromanovsky marked this pull request as ready for review May 13, 2026 17:25
@leoromanovsky leoromanovsky requested review from a team as code owners May 13, 2026 17:25
@leoromanovsky leoromanovsky requested review from dd-oleksii, greghuels and sameerank and removed request for a team and greghuels May 13, 2026 17:25
Copy link
Copy Markdown
Contributor

@sameerank sameerank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might help to include instructions like this for folks who may not be familiar

If this is a fresh clone, initialize submodules first:

git submodule update --init --recursive

perhaps in this section?

libdatadog/README.md

Lines 27 to 30 in 6e559d7

### Building
Build `libdatadog` as usual with `cargo build`.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants