Merge pull request #154 from compoundingtech/schickling-assistant/2026-08-03-149-ding-companion-lifecycle

fix: couple derived DING to agent lifecycle

Author: schickling

INVARIANTS.md

@@ -10,6 +10,7 @@ materialization, messaging, DING, or presence must preserve them.
 | **Transport-decoupled lifecycle** | Each task is isolated from a supervisor/transport process-group or cgroup cascade. | `tests/transport_isolation.rs`; `tests/transport_isolation_macos.rs` |
 | **Clean exec teardown** | Killing an exec task reaps its whole process group. | `tests/exec_backend.rs::exec_kill_reaps_the_whole_process_group_not_just_the_leader` |
 | **Bounded restart diagnostics** | Relaunching an exec task preserves the just-finished log as one prior generation while bounding retained diagnostics to current plus prior. Final retirement removes the PID and both logs. | `tests/exec_backend.rs::exec_restart_reap_keeps_bounded_diagnostics_and_final_remove_cleans_them`; `tests/run.rs::up_once_finally_removes_dead_retired_tasks_without_restarting_them` |
+| **Derived companion lifecycle** | A generated DING starts only with an eligible canonical agent, is suppressed or stopped while that target is held, unavailable, or terminally parked, and remains coupled without changing explicit sibling task behavior. Healthy compact startup still launches both tasks in one pass, retirement stops both, and targeted reconciliation holds a missing generated DING rather than broadening to its agent. | `tests/run.rs::fresh_compact_agent_launches_with_its_derived_ding`; `tests/run.rs::absent_adopt_only_compact_agent_holds_its_derived_ding`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::failed_compact_agent_restart_stops_its_live_derived_ding`; `tests/run.rs::failed_compact_agent_reap_stops_its_live_derived_ding`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::parked_compact_agent_does_not_relaunch_its_exited_derived_ding`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_ding`; `tests/run.rs::selected_missing_derived_ding_is_held_without_broadening_to_its_agent`; `tests/run.rs::up_once_collects_spawn_errors_without_aborting` |
 | **Exactly-once-safe native bus** | Messages use stable `<unix-ms>-<rand6>.md` files. An archive filename is a durable receipt that shadows and cleans restored inbox replicas and makes repeated archive cleanup idempotent. | `src/message.rs::filename_grammar`; `src/message.rs::archive_receipt_suppresses_and_idempotently_cleans_a_restored_inbox_copy`; `tests/message.rs` |
 | **Fail-closed observed native DING** | Each unread message becomes one normalized `[DING]` frame. Fresh delivery records ownership, then preserves the one combined bracketed-paste, 0.5 second delay, and Return transaction. PTY and Return success are transport only: `Delivered` additionally requires adapter classification of the expected notice text in a submitted-prompt or queued-message pattern while the lowest live composer is empty or an accepted idle placeholder. Retry never re-pastes and may send one bare Return only after two adjacent `RetainedSafe` observations. A maintained adapter's positive `NotRetained` observation releases only an already archived staged head; unread, blocked, timed-out, errored, unknown, and unrecognized states retain ownership and later FIFO work remains blocked. Ownership prevents duplicate paste across command failures, receipt ambiguity, archive races, and restart adoption without letting a vanished archived head block FIFO indefinitely. Startup backlog otherwise becomes one generic recovery DING; new arrivals remain FIFO; `busy` delivers immediately; only fresh `dnd` defers. | `src/ding/mod.rs::poke_text_normalizes_and_bounds_untrusted_fields`; `src/ding/mod.rs::malicious_controls_cannot_escape_the_single_paste_frame`; `src/ding/mod.rs::pty_delivery_uses_face607_delay_order_and_seconds`; `src/ding/mod.rs::maintained_composer_classifiers_require_exact_idle_state`; `src/ding/mod.rs::successful_transport_with_retained_or_unproven_pixels_is_not_delivered`; `src/ding/mod.rs::ambiguous_transport_receipt_and_retry_errors_retain_staged_ownership`; `src/ding/mod.rs::adapter_recognized_notice_with_an_empty_live_composer_is_a_positive_receipt`; `src/ding/mod.rs::staged_retry_submits_only_retained_safe_and_requires_a_receipt`; `src/ding/mod.rs::staged_retry_keeps_unproven_and_retained_blocked_owned`; `src/ding/mod.rs::staged_ownership_survives_archive_and_never_repastes`; `src/ding/mod.rs::archived_not_retained_releases_fifo_without_repasting_owned_notice`; `src/ding/mod.rs::unread_not_retained_keeps_fifo_ownership_without_repasting`; `src/ding/mod.rs::pty_commands_have_a_real_outer_timeout`; `src/ding/mod.rs::session_watch_has_startup_grace_debounce_and_live_reset`; `src/ding/mod.rs::new_arrivals_is_fifo_and_archive_receipts_prevent_reding`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry`; `src/ding/mod.rs::startup_recovery_notice_retries_in_memory`; `src/ding/mod.rs::startup_backlog_gets_one_generic_recovery_then_new_arrivals_poke` |
 | **Mutation-only filesystem wakeups** | Supervisor and DING filesystem watchers ignore read/open access events and wake early only for create, modify, rename, or remove events. Their own catalog and inbox reads therefore cannot bypass the bounded timer cadence or form a Linux inotify CPU loop. | `src/watch.rs::only_mutations_wake_watch_loops`; `src/watch.rs::linux_reads_are_silent_but_real_mutations_wake`; `src/ding/mod.rs::idle_ding_does_not_spin_on_its_own_inbox_reads`; `src/run.rs::idle_supervisor_does_not_spin_on_its_own_catalog_reads` |

README.md

@@ -174,7 +174,9 @@ agent "<identity>" {
 For a zero-interruption migration, add `lifecycle "adopt-only"` to the compact
 agent or to an explicit `pty`/`exec` task. st2 adopts that task when its current
 generation is alive. If the generation is dead or absent, st2 reports the task
-as `held` and does not remove or launch anything:
+as `held` and does not remove or launch that task. A compact `ding` is derived
+from the generated agent task: while that agent is held, st2 suppresses a
+missing DING and stops an exact generated DING proved live:
 
 ```kdl
 agent "<identity>" {
@@ -301,6 +303,9 @@ st2 up --catalog "$CATALOG" --host <host> --once --task <host.agent.task>
 ```
 
 Unknown, ambiguous, and wrong-host task selectors refuse before workspace writes or PTY inspection.
+A selected generated DING that is dead or absent is reported as `held`, because starting its
+canonical agent would broaden the exact-task operation. Explicit sibling tasks remain independently
+selectable.
 
 `st2 doctor` accepts the absence of a live host lock as the normal manual/`--once` mode. For a
 resident `st2 up` deployment, use `st2 doctor --require-supervisor` to make a missing loop fail the

docs/vrs/02-agent-spec/spec.md

@@ -113,7 +113,12 @@ evidence: [materializer](../../../src/materialize.rs).
 <h3 id="f09">F09 Task set: <code>pty</code>, <code>exec</code>, or compact <code>ding</code></h3>
 
 Add only the unique missing child. Remove and clean only an old child with exact
-ownership proof. A compact DING task uses the same rule. Do not change siblings.
+ownership proof. A compact DING is a derived child: it starts only after its
+canonical agent is already live or starts successfully in the same pass. When
+that target is held, fails to restart, or is terminally parked, do not launch
+the derived child and stop an exact generated child proved live. Explicit
+sibling tasks, including an authored `exec "ding"`, remain independent. Do not
+change unrelated siblings.
 
 Authoring: [pinned compact and explicit tasks][evals-tasks]. st2 source:
 [`Task` and `TaskKind`](../../../crates/agent-spec/src/spec.rs). Evidence:
@@ -148,8 +153,10 @@ Evidence: [spawn construction](../../../src/run.rs).
 
 Agent or task `keep`, restart `attempts`, `interval`, `delay`, and `mode`, and
 task `lifecycle` are future policy. Adopt healthy work. `adopt-only` holds absent
-or dead work; `service` reconciles it normally. Invalid policy refuses changes
-to the related agent and tasks.
+or dead work; `service` reconciles it normally. A generated companion follows
+the canonical agent's effective eligibility: `adopt-only` holds it, and
+exhausting a fail-mode restart policy stops or suppresses it. Invalid policy
+refuses changes to the related agent and tasks.
 
 Authoring: [pinned complete declaration][evals-fields]. The
 [pinned explicit-task list][evals-task-fields] and st2 `9887b28` predate task
@@ -170,8 +177,9 @@ Authoring: [pinned complete declaration][evals-fields]. st2 source:
 <h3 id="f14">F14 Compact agent fields</h3>
 
 Compact `command`, `argv`, `env`, `lifecycle`, and `ding` convert to the
-generated agent PTY and sidecar. The tasks use F09, F11, and F12. Compact syntax
-adds no other behavior.
+generated agent PTY and derived sidecar. The tasks use F09, F11, and F12;
+`ding` carries the dependency on the generated agent task described there.
+Compact syntax adds no other behavior.
 
 Authoring: [pinned compact tasks][evals-tasks]. That document and st2 `9887b28`
 predate compact `argv` and `lifecycle`. Current st2 source:

docs/vrs/spec.md

@@ -430,7 +430,12 @@ validate ──► materialize ──► host-local st2 scheduler/reconciler
   loop is deterministic; exactly one declared root agent provides intelligent
   host-local supervision, bounded recovery, and escalation. Filesystem reads
   never wake reconciliation; only create, modify, rename, or remove events may
-  wake it before the bounded timer.
+  wake it before the bounded timer. A generated companion is eligible only
+  while its canonical agent task is eligible. Healthy startup launches the
+  agent first and then its missing companions in the same pass. Holding or
+  failing to restart the agent, or terminally parking it, suppresses companion
+  launch and stops an exact generated companion proved live; explicitly
+  authored sibling tasks remain independent.
 - **R06:** st2 passes the complete effective task definition to the underlying
   launcher so manual and supervised restarts are equivalent. Harness readiness
   that depends on a dynamically selected account belongs to that declared
@@ -613,7 +618,10 @@ task before writing and renders only its owning agent. `st2 up --once --task
 PTY/exec state and executes a plan containing only that task. Unknown,
 ambiguous, and wrong-host selectors refuse before writes or runner inspection;
 unrelated discovery diagnostics remain visible without preventing the selected
-owner/task path.
+owner/task path. A live generated companion may be adopted or retired as that
+exact selected task. An active dead or absent generated companion is held: the
+bounded pass cannot start its canonical agent without broadening the selector.
+Explicitly authored sibling tasks retain ordinary selected-task behavior.
 
 `st2 up --materialize-only --agent <id>` remains the agent-wide rendering
 selector. Targeted task reconciliation is intentionally bounded to `--once`;

src/exec_backend.rs

@@ -933,6 +933,7 @@ mod generation_observation_tests {
             pty_id: id.to_string(),
             bus_id: "host.test".to_string(),
             name: "probe".to_string(),
+            derived: false,
             launch: TaskLaunch::Shell("sleep 30".to_string()),
             cwd: None,
             workspace: None,

src/reconcile.rs

@@ -4,9 +4,9 @@
 //!
 //! Pure and side-effect-free, so it is exhaustively unit-testable; execution lives behind backends
 //! (the `pty` CLI for `pty` tasks, direct process supervision for terminal-free `exec` tasks). st2
-//! reconciles at the **task** level — an agent declares several tasks (its harness pty + its ding
-//! exec) and each is kept running independently; a spec with one live task and one dead task is a
-//! launch of just the missing one.
+//! reconciles at the **task** level — explicitly authored sibling tasks remain independent. A
+//! generated companion is instead eligible only with its canonical agent task and is stopped when
+//! that target is held or terminally parked.
 
 use std::collections::BTreeMap;
 use std::collections::HashMap;
@@ -46,6 +46,8 @@ pub struct TaskTarget {
     pub bus_id: String,
     /// The task name (`agent`, `ding`, …).
     pub name: String,
+    /// Generated from another task rather than authored as an independent sibling.
+    pub derived: bool,
     /// How to launch the task: shell source or a direct program argument vector.
     pub launch: TaskLaunch,
     /// Declared working dir; `None` → default to `workspace`, else the spec dir (resolved at spawn).
@@ -131,9 +133,12 @@ pub enum TaskLaunch {
 pub struct Launch<'a> {
     pub spec: &'a AgentSpec,
     pub tasks: Vec<TaskTarget>,
+    /// Exact derived task IDs proved live in the same inventory snapshot. Execution stops these if
+    /// the canonical agent becomes terminal while applying this launch.
+    pub live_derived: Vec<String>,
 }
 
-/// An agent to tear down (retired) — the live task ids to kill.
+/// Exact live task IDs to stop because their owner retired or their derived target is ineligible.
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub struct Teardown<'a> {
     pub spec: &'a AgentSpec,
@@ -213,6 +218,10 @@ pub fn reconcile_selected<'a>(
         }
         return Ok(plan);
     }
+    if task.derived && !actual.is_some_and(|session| session.alive) {
+        plan.held.push(runtime);
+        return Ok(plan);
+    }
     let launch = match (&task.command, &task.argv) {
         (Some(command), None) => TaskLaunch::Shell(command.clone()),
         (None, Some(argv)) => TaskLaunch::Argv(argv.clone()),
@@ -236,6 +245,7 @@ pub fn reconcile_selected<'a>(
         pty_id: runtime.clone(),
         bus_id: bus_id.clone(),
         name: task.name.clone(),
+        derived: task.derived,
         launch,
         cwd: task.cwd.clone(),
         workspace: owner.workspace.clone(),
@@ -263,17 +273,20 @@ pub fn reconcile_selected<'a>(
             plan.launch.push(Launch {
                 spec: owner,
                 tasks: vec![target],
+                live_derived: Vec::new(),
             });
         }
         _ => plan.launch.push(Launch {
             spec: owner,
             tasks: vec![target],
+            live_derived: Vec::new(),
         }),
     }
     Ok(plan)
 }
 
 /// The state of a declared task's session in the ACTUAL world.
+#[derive(Clone, Copy)]
 enum SessionState {
     Alive,
     Dead,
@@ -372,6 +385,7 @@ pub fn reconcile<'a>(
                         pty_id: pty_id.clone(),
                         bus_id: bus_id.clone(),
                         name: t.name.clone(),
+                        derived: t.derived,
                         launch,
                         cwd: t.cwd.clone(),
                         workspace: spec.workspace.clone(),
@@ -387,11 +401,42 @@ pub fn reconcile<'a>(
 
         debug_assert!(!targets.is_empty());
 
+        let agent_eligible = targets
+            .iter()
+            .find(|(target, _)| target.name == "agent" && !target.derived)
+            .is_some_and(|(target, lifecycle)| match session_state(&by_id, &target.pty_id) {
+                SessionState::Alive => true,
+                SessionState::Dead => {
+                    !target.keep && *lifecycle == TaskLifecycle::Service
+                }
+                SessionState::Absent => *lifecycle == TaskLifecycle::Service,
+            });
         let mut to_launch = Vec::new();
+        let mut live_derived = Vec::new();
+        let mut ineligible_derived = Vec::new();
+        let mut derived_cleanup = false;
         let held_before = plan.held.len();
         for (target, lifecycle) in targets {
-            match session_state(&by_id, &target.pty_id) {
+            let state = session_state(&by_id, &target.pty_id);
+            if target.derived && !agent_eligible {
+                match state {
+                    SessionState::Alive => {
+                        ineligible_derived.push(target.pty_id);
+                        derived_cleanup = true;
+                    }
+                    SessionState::Dead if !target.keep => {
+                        plan.gc.push(target.pty_id);
+                        derived_cleanup = true;
+                    }
+                    SessionState::Dead | SessionState::Absent => {}
+                }
+                continue;
+            }
+            match state {
                 SessionState::Alive => {
+                    if target.derived {
+                        live_derived.push(target.pty_id.clone());
+                    }
                     let actual = sessions_by_id
                         .get(target.pty_id.as_str())
                         .expect("alive state has a session");
@@ -417,13 +462,20 @@ pub fn reconcile<'a>(
                 SessionState::Absent => to_launch.push(target),
             }
         }
+        if !ineligible_derived.is_empty() {
+            plan.teardown.push(Teardown {
+                spec,
+                pty_ids: ineligible_derived,
+            });
+        }
 
-        if to_launch.is_empty() && plan.held.len() == held_before {
+        if to_launch.is_empty() && plan.held.len() == held_before && !derived_cleanup {
             plan.adopt.push(spec);
         } else if !to_launch.is_empty() {
             plan.launch.push(Launch {
                 spec,
                 tasks: to_launch,
+                live_derived,
             });
         }
     }