enhancement(core): add configurable per-child restart types for supervisors

[tobz]

Summary

This PR adds support for adjusting the restart type of child processes.

In Erlang/OTP, supervisors expose a lot more control over how child processes are supervised, up to and including whether or not to actually restart the process. Our current implementation of Supervisor hardcodes the behavior of always restarting a child process, but this is not flexible enough for our needs.

This PR introduces a new RestartType enum which encodes the possible restart types available: permanent (always restart; default), transient (restart if process exited with an error), and temporary (never restart). This mimics the restart types exposed in Erlang/OTP. The corresponding worker restart logic has been updated to handle and respect these new restart types. We've also exposed some new machinery in ChildSpecification to allow specifying a non-default restart type, and to do so safely, we've used some typestate magic so that people can't do incorrect things like change the restart type for a supervisor, and so on.

Support for restart types sets us up for a number of incremental improvements towards a fuller feature set in supervisors, including significant children and dynamically added children.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

How did you test this PR?

New and existing unit tests.

References

DADP-2

[tobz]

• chore(core): unify how we define identifiers for subsystems/components #2029
• chore(core): integrate component tasks via supervision trees #1942
• enhancement(core): add support for dynamically added child processes in Supervisor #1874
• enhancement(core): add configurable per-child restart types for supervisors #1873 👈 (View in Graphite)
• chore(core): clean up a bunch of code smells around supervisable worker implementations #1860
• chore(common): unify multiple shutdown synchronization types #1856
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: 35ec839 · Comparison: 36c4cc9 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.14 MiB (baseline) vs 40.00 MiB (comparison)
Size Change: -145.62 KiB (-0.35%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
core	-45.42 KiB	1334
figment	-36.23 KiB	130
prost	-23.05 KiB	61
tower	+21.11 KiB	139
alloc	+19.44 KiB	422
tonic	-15.70 KiB	204
axum	-13.14 KiB	150
hashbrown	+9.05 KiB	82
saluki_core::runtime::supervisor	-6.90 KiB	156
saluki_core::topology::built	-6.72 KiB	5
std	-6.69 KiB	73
&mut serde_json	-6.65 KiB	14
datadog_protos::agent_include::datadog	-6.50 KiB	14
resource_accounting::groups::Tracked	-6.46 KiB	2
saluki_common::cache::Cache<K,V,W,H>	-5.71 KiB	3
[Unmapped]	+5.24 KiB	1
saluki_api::DynamicRoute::http	+4.92 KiB	2
quick_cache	+4.53 KiB	14
saluki_core::topology::interconnect	+4.01 KiB	12
serde_core	-4.00 KiB	39

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +737% +15.0Ki  +779% +15.0Ki    saluki_core::topology::built::ComponentInterconnects::from_graph::hc0a708c15e73d666
  [NEW] +9.96Ki  [NEW] +9.86Ki    saluki_core::runtime::supervisor::Supervisor::spawn_child::h0c9f27f10c42604b
  [NEW] +9.93Ki  [NEW] +9.77Ki    saluki_core::runtime::supervisor::Supervisor::run_inner::_{{closure}}::haee8906d37dd6144
  [NEW] +7.66Ki  [NEW] +7.53Ki    saluki_core::runtime::supervisor::WorkerState::shutdown_workers::_{{closure}}::h207bbee351259e71
  [NEW] +7.37Ki  [NEW] +7.23Ki    _<tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::hf1213bdc5b863f31
  [NEW] +6.71Ki  [NEW] +6.56Ki    _<core::marker::PhantomData<T> as serde_core::de::DeserializeSeed>::deserialize::h059d34dc1479ec6b
  [NEW] +6.58Ki  [NEW] +6.44Ki    saluki_core::runtime::supervisor::WorkerState::shutdown_workers::_{{closure}}::hb530e010aa40c8f0
  [NEW] +5.40Ki  [NEW] +5.14Ki    agent_data_plane::components::ottl_filter_processor::config::_::_<impl serde_core::de::Deserialize for agent_data_plane::components::ottl_filter_processor::config::OttlFilterConfig>::deserialize::hdd3789c2aade9d43
  +181% +5.24Ki  [ = ]       0    [Unmapped]
  [DEL] -5.29Ki  [DEL] -5.19Ki    saluki_common::cache::Cache<K,V,W,H>::insert::hdfb73f67ef88d057
  [DEL] -5.39Ki  [DEL] -5.24Ki    _<figment::value::magic::RelativePathBuf as figment::value::magic::Magic>::deserialize_from::h7fbda6ef77e9f8af
  [DEL] -6.44Ki  [DEL] -6.28Ki    _<figment::value::de::ConfiguredValueDe<I> as serde_core::de::Deserializer>::deserialize_struct::h1416187a5e0bf26c
  [DEL] -6.54Ki  [DEL] -6.39Ki    _<resource_accounting::groups::Tracked<Inner> as core::future::future::Future>::poll::hf185c41be28e49f7
  [DEL] -6.64Ki  [DEL] -6.50Ki    saluki_core::runtime::supervisor::WorkerState::shutdown_workers::_{{closure}}::h2eaa74c3095947a0
  [DEL] -6.70Ki  [DEL] -6.54Ki    _<&mut serde_json::de::Deserializer<R> as serde_core::de::Deserializer>::deserialize_struct::h62f2718d6db6fc06
  [DEL] -6.77Ki  [DEL] -6.66Ki    saluki_core::runtime::supervisor::Supervisor::spawn_child::h7ef0c7c2bb619654
  [DEL] -6.87Ki  [DEL] -6.72Ki    _<tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h9ec4e924070543f9
  [DEL] -7.76Ki  [DEL] -7.62Ki    saluki_core::runtime::supervisor::Supervisor::as_nested_process::_{{closure}}::h41e16f23e6b2cf11
  [DEL] -14.7Ki  [DEL] -14.5Ki    saluki_core::runtime::supervisor::WorkerState::shutdown_workers::_{{closure}}::ha7d85b13d5343348
  [DEL] -21.4Ki  [DEL] -21.3Ki    saluki_core::topology::built::ComponentInterconnects::generate_interconnects::h7c528a9174c5a012
  -1.2%  -124Ki  -1.4%  -123Ki    [8704 Others]
  -0.4%  -145Ki  -0.5%  -149Ki    TOTAL

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 0e0370f5-99f3-4597-880e-eea279f3f660
Baseline: a6137f5b · Comparison: e678e20a · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
otlp_ingest_logs_5mb_memory (ignored)	memory	⚪ +6.29	metrics profiles logs
dsd_uds_512kb_3k_contexts_cpu (erratic)	cpu	⚪ +4.28	metrics profiles logs
dsd_uds_1mb_3k_contexts_cpu (erratic)	cpu	⚪ +2.88	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic)	cpu	⚪ +2.44	metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput	throughput	⚪ -2.25	metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic)	cpu	⚪ +1.04	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic)	cpu	⚪ +0.98	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput	throughput	⚪ -0.27	metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic)	cpu	⚪ +0.22	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ +0.21	metrics profiles logs
otlp_ingest_traces_5mb_memory	memory	⚪ +0.19	metrics profiles logs
otlp_ingest_traces_5mb_throughput	throughput	⚪ -0.04	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory	⚪ +0.02	metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput	throughput	⚪ -0.02	metrics profiles logs
otlp_ingest_metrics_5mb_throughput	throughput	⚪ -0.01	metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput	throughput	⚪ -0.00	metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput	throughput	⚪ +0.00	metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput	throughput	⚪ +0.00	metrics profiles logs
dsd_uds_100mb_3k_contexts_memory	memory	⚪ -0.01	metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored)	throughput	⚪ +0.03	metrics profiles logs
dsd_uds_512kb_3k_contexts_memory	memory	⚪ -0.07	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput	throughput	⚪ +0.07	metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory	memory	⚪ -0.12	metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory	memory	⚪ -0.19	metrics profiles logs
dsd_uds_10mb_3k_contexts_memory	memory	⚪ -0.20	metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic)	cpu	⚪ -0.21	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ -0.25	metrics profiles logs
dsd_uds_500mb_3k_contexts_memory	memory	⚪ -0.53	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.60	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.66	metrics profiles logs
dsd_uds_1mb_3k_contexts_memory	memory	⚪ -0.75	metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic)	cpu	⚪ -1.33	metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored)	cpu	⚪ -1.75	metrics profiles logs
otlp_ingest_metrics_5mb_memory	memory	⚪ -2.03	metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic)	cpu	🟢 -7.41	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 133 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 42.2 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 64.7 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 191 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 28.2 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

[chatgpt-codex-connector]

💡 Codex Review

saluki/lib/saluki-core/src/runtime/supervisor.rs

Line 617 in a87eb98

self.spawn_all_children(&mut worker_state)?;

Avoid relaunching cleanly exited transient children

With RestartMode::OneForAll, if a RestartType::Transient child has already exited with Ok(()), the supervisor leaves it stopped as intended, but a later failure from any sibling reaches this spawn_all_children() call and starts every static child spec again. That means a transient one-shot child that completed normally can be relaunched even though RestartType::Transient documents that normal exits are intentional and not restarted; this can rerun completed work whenever a sibling trips one-for-all restart.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Copilot]

Pull request overview

This PR enhances saluki-core’s runtime supervision framework by adding Erlang/OTP-style per-child restart policies, allowing supervisors to selectively restart children based on how they exited.

Changes:

• Introduces RestartType (Permanent / Transient / Temporary) and exports it via runtime.
• Refactors ChildSpecification into a typestate-based API to allow configuring restart type for workers while preventing invalid configurations for nested supervisors.
• Updates supervisor run-loop logic and adds unit tests covering new restart-type behavior (including “no workers left” idling behavior).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
lib/saluki-core/src/runtime/supervisor.rs	Adds typestate-based child specs, stores per-child RestartType, updates restart/idling logic, and adds restart-type unit tests.
lib/saluki-core/src/runtime/restart.rs	Adds the RestartType enum and internal restart-eligibility helper.
lib/saluki-core/src/runtime/mod.rs	Re-exports new runtime API surface (RestartType, ChildSpecification, typestate markers).
.vale/styles/config/vocabularies/technical/accept.txt	Adds “typestate” to the Vale technical vocabulary.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.