chore(components, core, io): cutover Datadog Traces encoder and Checks IPC source to full supervision

[tobz]

Summary

This PR is the first of many (hopefully just a few) for converting components to spawn all of their child tasks under their dedicated supervisor to fully unify task execution in Saluki under the supervision tree model.

This PR is meant to get our feet wet: a simple component (Datadog Traces encoder) and a more complex one (Checks IPC source).

The Datadog Traces encoder conversion exercises spawning a worker based purely on an async function (which is the "simple" model) as well as executing it on the worker pool, rather than whatever runtime the component supervisor is attached to.

The Checks IPC source conversion exercises spawning a supervisable worker, which is a new addition in this PR: I realized that we're not always going to want to just spawn async fns that can't ever be restarted, and that it was a good opportunity to update ComponentSpawner to support that and add a new gRPC server primitive which implements Supervisable natively so we could exercise the new support for spawning Supervisables.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

How did you test this PR?

Existing tests.

References

DADP-2

[tobz]

• chore(components, core, io): cutover Datadog Traces encoder and Checks IPC source to full supervision #2342 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: 8d372a5 · Comparison: 7d7db10 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 41.27 MiB (baseline) vs 41.66 MiB (comparison)
Size Change: +398.47 KiB (+0.94%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
core	+173.01 KiB	13121
saluki_common::resource_tracking::groups	-140.09 KiB	39
tracing	+131.47 KiB	127
hyper	+50.45 KiB	535
figment	+34.62 KiB	333
[sections]	+34.39 KiB	9
serde_core	-24.31 KiB	980
tonic	-20.39 KiB	410
axum	+19.94 KiB	521
saluki_components::transforms::dogstatsd_mapper	+19.86 KiB	20
h2	+18.52 KiB	811
prost	-18.30 KiB	493
http_body_util	+18.22 KiB	226
anon.6c8cc82583d787f2710e1f8f86e78987.13.llvm.13314861055718887123	-17.17 KiB	1
anon.023b4777736ee221e94e1131aa3c0740.13.llvm.1302689955535841485	+17.00 KiB	1
hyper_util	+16.59 KiB	163
saluki_io::net::server	+15.93 KiB	37
tokio	+12.31 KiB	4515
saluki_core::runtime::workers	+12.30 KiB	6
anon.eaa35ebdc5998855717b6f667ad22404.791.llvm.3148331925466594304	-11.93 KiB	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +1.7%  +326Ki  +1.5%  +228Ki    [46573 Others]
  [NEW] +60.5Ki  [NEW] +60.3Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::h382682f7f52d5bea
  [NEW] +42.2Ki  [NEW] +42.1Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h8ff13e0bab4ddc8e
  [NEW] +41.8Ki  [NEW] +41.6Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::h777d38f79b4f90c8
  [NEW] +40.5Ki  [NEW] +40.3Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h2431c8b4674292e5
  [NEW] +37.5Ki  [NEW] +37.3Ki    _<saluki_components::transforms::apm_stats::ApmStats as saluki_core::components::transforms::Transform>::run::_{{closure}}::h5577cdbd4b55abed
  [NEW] +36.8Ki  [NEW] +36.6Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::h87d4cd12e4183c06
  [NEW] +36.6Ki  [NEW] +36.4Ki    saluki_components::sources::otlp::metrics::translator::OtlpMetricsTranslator::translate_metrics::h312a34406c2e2021
  [NEW] +31.9Ki  [NEW] +31.6Ki    _<saluki_components::sources::dogstatsd::_::<impl serde_core::de::Deserialize for saluki_components::sources::dogstatsd::DogStatsDConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h1fb3499e177dc008
  [NEW] +30.6Ki  [NEW] +30.5Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::hb824a5fdcb8383f2
  [NEW] +28.5Ki  [NEW] +28.3Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h86d40d03ebf53146
  [NEW] +26.8Ki  [NEW] +26.6Ki    datadog_agent_commons::ipc::client::RemoteAgentClient::from_client_configuration::_{{closure}}::_{{closure}}::_{{closure}}::h9f0b2492dc22c2ef
  [DEL] -26.7Ki  [DEL] -26.5Ki    core::ptr::drop_in_place<agent_data_plane::cli::run::handle_run_command::{{closure}}>::hac8ba744a6354ad7
  [DEL] -28.5Ki  [DEL] -28.3Ki    agent_data_plane::dogstatsd_contexts::artifact::for_each_record::h39b2832027986acc
  [DEL] -30.6Ki  [DEL] -30.4Ki    agent_data_plane::cli::dogstatsd::run_dogstatsd_command::_{{closure}}::h8e6a26bba5766166
  [DEL] -35.5Ki  [DEL] -35.2Ki    _<saluki_components::sources::dogstatsd::_::<impl serde_core::de::Deserialize for saluki_components::sources::dogstatsd::DogStatsDConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h6a606a7931ae90c3
  [DEL] -38.5Ki  [DEL] -38.3Ki    _<saluki_components::transforms::aggregate::Aggregate as saluki_core::components::transforms::Transform>::run::_{{closure}}::h5212ad494e6e6d30
  [DEL] -39.5Ki  [DEL] -39.3Ki    _<saluki_components::forwarders::otlp::OtlpForwarder as saluki_core::components::forwarders::Forwarder>::run::_{{closure}}::hcd2e7e0448ed9644
  [DEL] -40.3Ki  [DEL] -40.2Ki    agent_data_plane::cli::run::create_topology::_{{closure}}::h23953cd93c271a64
  [DEL] -42.2Ki  [DEL] -42.1Ki    agent_data_plane::cli::run::handle_run_command::_{{closure}}::h8c5bc31d01253285
  [DEL] -60.1Ki  [DEL] -59.9Ki    saluki_components::common::datadog::io::run_endpoint_io_loop::_{{closure}}::hd9fc277b29cf0b9d
  +0.9%  +398Ki  +0.8%  +299Ki    TOTAL

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7d7db102ca

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Keep Checks IPC gRPC on the worker pool

When Checks IPC receives large or continuous payloads, this now runs the Tonic server and its request handlers on the component supervisor's runtime because supervisable defaults to that runtime. The previous implementation explicitly spawned Server::serve on global_thread_pool(), so this cutover moves protobuf decoding and check_data_to_event processing onto the runtime that drives topology supervision and I/O, allowing Checks traffic to increase scheduling latency for unrelated components. Add .on_worker_pool() before spawning to preserve the previous isolation.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Correct the interruptible task restart documentation

This public API documentation says interruptible tasks restart permanently, but interruptible constructs a OneShot builder and ChildBuilder::one_shot uses ChildSpecification::one_shot_worker, which sets RestartType::Temporary. Callers relying on the stated policy could expect a failed background task to recover when it will instead disappear permanently, so document the actual temporary behavior.

Useful? React with 👍 / 👎.

[datadog-datadog-prod-us1-2]

The new gRPC primitive makes saluki-io directly use tonic’s router/server APIs without enabling those features. Workspace feature unification can hide the defect, but package-scoped and downstream builds of saluki-io will fail to compile.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 7d7db10 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[datadog-datadog-prod-us1-2]

Declare tonic server features in saluki-io

Consumers that build saluki-io without saluki-components cannot compile the crate.

Assertion details

• Input: A package-scoped or downstream build where no other dependency enables tonic's router and server features.
• Expected: Update lib/saluki-io/Cargo.toml to enable tonic's router and server features directly, matching the crate's API usage. This requires a manifest change outside the relevant diff hunk, so no inline suggestion is safe.
• Actual: The new module imports tonic::server::NamedService, tonic::service::Routes, and transport server APIs, but saluki-io declares tonic = { workspace = true } while workspace defaults are disabled. Workspace-wide builds can mask this through saluki-components, which enables router and server, but package-scoped or downstream builds cannot resolve these APIs.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 88fa846a-fe70-4681-8ff1-4b7b5c09b0cc
Baseline: 8d372a51 · Comparison: 7d7db102 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_low	memory	⚪ +0.52	metrics profiles logs
quality_gates_rss_idle	memory	⚪ +0.24	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ +0.23	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.00	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ -0.99	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 231 MiB ≤ 250 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 51.5 MiB ≤ 60 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 90.8 MiB ≤ 100 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 390 MiB ≤ 420 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 31.9 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression. Experiments tagged (no analysis) show ⚠️ n/a: SMP ran them but produced no analysis, usually because a replicate failed and exhausted its retries. Check the SMP report for that experiment's replicate failures.