chore(config): integrate and test the config system

[webern]

Human Summary

Integrates the typed config system into agent-data-plane even though no components are yet reading from it. We decided to take a migration approach to this, so components will be cut over to using typed config in isolated PRs where we can focus on correctness for each component.

This PR introduces a privileged API endpoint /config/internal which is a serialization of SalukiConfig. This is going to be important for end-to-end integration testing, and the first such test is added in this PR.

Also noteworthy is that all integration and correctness tests are passing here, which was not the case prior to #2093's improvement of how the type config system handles environment variables.

AI Summary

Integrate the typed configuration system into the Agent Data Plane runtime and expose its current value through the internal configuration endpoint.

• Load the configuration system after the initial configuration snapshot is available.
• Thread the configuration-system handle through runtime setup and the internal supervisor.
• Serve the current configuration through /config/internal.
• Use fallback environment-overlay behavior so Agent-provided values remain authoritative.
• Add integration coverage for runtime configuration updates through the internal endpoint.

Change Type

• Non-functional (chore, refactoring, docs)

How did you test this PR?

• make build-schema-overlay
• make fmt
• make check-docs
• make check-all
• make test

All checks passed; 1,830 tests passed and 31 were skipped.

References

Stacked on #2093.

[datadog-datadog-prod-us1]

⚠️ Warnings

🚦 2 Pipeline jobs failed

DataDog/saluki | test-integration     

DataDog/saluki | Ubuntu base image check     

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1f0ed22 | Docs | Datadog PR Page | Give us feedback!}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: f6a508e · Comparison: 1f0ed22 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 39.39 MiB (baseline) vs 40.86 MiB (comparison)
Size Change: +1.47 MiB (+3.73%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
core	+270.62 KiB	4397
serde_json	+258.04 KiB	366
datadog_agent_config::generated::datadog_configuration	+109.28 KiB	50
serde_core	+98.26 KiB	478
agent_data_plane_config_system::saluki_env_overlay::PathRecorder	+69.56 KiB	25
alloc	+67.72 KiB	736
[sections]	+63.69 KiB	9
tokio	+34.29 KiB	1192
serde_yaml	+27.63 KiB	63
hashbrown	+25.13 KiB	333
agent_data_plane_config_system::translators::datadog_translator	+22.07 KiB	22
anon.1c766fa05dd13eba06da95c69824d6af.983.llvm.9357882842819282106	+17.19 KiB	1
saluki_core::runtime::process	-17.14 KiB	5
anyhow	+17.00 KiB	527
anon.4f47395eaac7e6a86c9111625e37e6d6.395.llvm.10306485434466900905	+16.92 KiB	1
figment	+16.79 KiB	186
axum	+16.65 KiB	155
anon.e8497720c4a011bff86aa8f1d6f5d84b.7.llvm.12958167161027842560	-16.12 KiB	1
datadog_agent_config::generated::witness	+13.65 KiB	1
tracing	+13.42 KiB	8

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
   +21% +1.34Mi   +21%  +950Ki    [15875 Others]
  [NEW] +50.9Ki  [NEW] +50.6Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::hd1d904526519d139
  [NEW] +43.8Ki  [NEW] +43.6Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::ha5ca38efb1a9f33f
  +3.2% +35.9Ki  +3.2% +35.9Ki    [section .gcc_except_table]
  [NEW] +30.7Ki  [NEW] +30.5Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::from_client_configuration::_{{closure}}::_{{closure}}::_{{closure}}::h97868b07fa692b3b
  [NEW] +30.6Ki  [NEW] +30.4Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::h14f8748dfe79d0fb
  [NEW] +29.2Ki  [NEW] +29.1Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::hb36732e5585105ab
  [NEW] +24.9Ki  [NEW] +24.7Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::h4e6044888b2bd00a
  [NEW] +21.5Ki  [NEW] +21.4Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h69081d15e29eb4bb
  [NEW] +21.3Ki  [NEW] +21.1Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::h2d1251391c27a303
  [NEW] +20.2Ki  [NEW] +20.1Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::h2f3a47ff8d1c568c
  [NEW] +18.7Ki  [NEW] +18.4Ki    _<datadog_agent_config::generated::datadog_configuration::_::<impl serde_core::de::Deserialize for datadog_agent_config::generated::datadog_configuration::DatadogConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_seq::hb5ee46c1eacda327
  [NEW] +18.3Ki  [NEW] +18.2Ki    agent_data_plane::main::_{{closure}}::h308a63b525bca90f
  [DEL] -20.0Ki  [DEL] -19.8Ki    agent_data_plane::internal::env::workload::build_collector::_{{closure}}::hb86ef6ad81fe7344
  [DEL] -20.6Ki  [DEL] -20.5Ki    agent_data_plane::internal::env::ADPEnvironmentProvider::from_configuration::_{{closure}}::h60e92a13bd2523e6
  [DEL] -21.0Ki  [DEL] -20.9Ki    agent_data_plane::cli::debug::handle_debug_command::_{{closure}}::h93f218996b92f607
  [DEL] -24.8Ki  [DEL] -24.7Ki    agent_data_plane::internal::remote_agent::run_remote_agent_registration_loop::_{{closure}}::hc92f998ffefc6e33
  [DEL] -25.9Ki  [DEL] -25.7Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::from_client_configuration::_{{closure}}::_{{closure}}::_{{closure}}::hbb3730e121df0e86
  [DEL] -28.0Ki  [DEL] -27.9Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h06e7f45a4338cfe6
  [DEL] -30.2Ki  [DEL] -30.1Ki    agent_data_plane::cli::dogstatsd::handle_dogstatsd_command::_{{closure}}::h1d3cca75feff50df
  [DEL] -40.6Ki  [DEL] -40.4Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h3ded2305a05b56fc
  +3.7% +1.47Mi  +3.2% +1.06Mi    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 6559f932-ac4a-40e5-bd6b-4750ebabe022
Baseline: f6a508ef · Comparison: 1f0ed22d · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (3)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_idle	memory	⚪ +2.89	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ +2.81	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ +1.64	metrics profiles logs

Bounds Checks: ✅ Passed (3)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 45.4 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 67 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 31.1 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[webern]

• chore(config): cutover adp bin components to typed config system #1995
• chore(config): cutover saluki-components to the new typed configuration system #1987
• chore(config): build the first components from the typed configuration model #1979
• chore(config): integrate and test the config system #2094 👈 (View in Graphite)
• chore(config): support environment variables in the config system #2093
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[datadog-datadog-prod-us1]

Static analysis only (build unavailable — hyper-http-proxy git dependency unreachable in sandbox). DD_MEMORY_LIMIT=10485760 arrives at ByteSize::from_str as the bare string "10485760" (no unit suffix) via EnvDecode::RawString; if bytesize 2.4.0 rejects unitless integer strings, ADP startup fails whenever memory_limit is set as a plain byte count.

ByteSize env-var bare-number decode is untested

ADP startup aborts for any deployment that sets DD_MEMORY_LIMIT to a plain byte count rather than a suffixed string like "10MB", which is a common scripting pattern.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit e1396dc · What is Autotest? · Any feedback? Reach out in #autotest}

[webern]

Static analysis only (build unavailable — hyper-http-proxy git dependency unreachable in sandbox). DD_MEMORY_LIMIT=10485760 arrives at ByteSize::from_str as the bare string "10485760" (no unit suffix) via EnvDecode::RawString; if bytesize 2.4.0 rejects unitless integer strings, ADP startup fails whenever memory_limit is set as a plain byte count.

ADP startup aborts for any deployment that sets DD_MEMORY_LIMIT to a plain byte count rather than a suffixed string like "10MB", which is a common scripting pattern.

@datadog-datadog-prod-us1 I'm 99.9% percent sure I fixed this in the preceding PR, #2093, which has been merged and this has been rebased to pick it up.