diff --git a/INVARIANTS.md b/INVARIANTS.md index 9f01451d..e5e232e9 100644 --- a/INVARIANTS.md +++ b/INVARIANTS.md @@ -13,7 +13,7 @@ materialization, messaging, DING, or presence must preserve them. | **Bounded restart diagnostics** | Relaunching an exec task preserves the just-finished log as one prior generation while bounding retained diagnostics to current plus prior. Final retirement removes the PID and both logs. | `tests/exec_backend.rs::exec_restart_reap_keeps_bounded_diagnostics_and_final_remove_cleans_them`; `tests/run.rs::up_once_finally_removes_dead_retired_tasks_without_restarting_them` | | **Derived companion lifecycle** | A generated DING or launched stream starts only with an eligible canonical agent, is suppressed or stopped while that target is held, suspended, unavailable, retired, or terminally parked, and remains coupled without changing explicit sibling task behavior. A stream lowers its authored adapter launch directly, parks and surfaces independently, and neither makes an otherwise empty agent runnable nor claims a delivery transport. Stream authoring is serialized, authority-scoped, source-preserving, and fail-closed for Nix ownership and invalid declarations. Healthy compact startup launches companions in one pass, and targeted reconciliation holds a missing generated companion rather than broadening to its agent. | `crates/agent-spec/tests/discovery.rs::streams_are_typed_and_only_launched_streams_lower_to_derived_exec_tasks`; `crates/agent-spec/tests/discovery.rs::stream_names_launches_and_task_collisions_fail_closed`; `src/agent_author.rs::stream_add_supports_external_command_and_argv_and_external_remove_is_idempotent`; `src/agent_author.rs::stream_authoring_enforces_authority_nix_ownership_and_canonical_validation`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_ding`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_ding`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_ding`; `tests/run.rs::selected_missing_derived_ding_is_held_without_broadening_to_its_agent`; `tests/run.rs::fresh_compact_agent_launches_with_its_derived_stream`; `tests/run.rs::retired_compact_agent_stops_agent_and_derived_stream`; `tests/run.rs::suspended_compact_agent_stops_its_derived_stream_without_touching_a_sibling`; `tests/run.rs::held_adopt_only_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::a_crash_looping_stream_parks_and_surfaces_without_disturbing_its_agent`; `tests/run.rs::parked_compact_agent_stops_its_live_derived_stream`; `tests/run.rs::selected_missing_derived_stream_is_held_without_broadening_to_its_agent`; `tests/run.rs::a_launched_stream_alone_is_rejected_before_reconciliation`; `tests/run.rs::a_stream_does_not_claim_a_delivery_transport` | | **Exactly-once-safe native bus** | Messages use stable `-.md` files. An archive filename is a durable receipt that shadows and cleans restored inbox replicas and makes repeated archive cleanup idempotent. | `src/message.rs::filename_grammar`; `src/message.rs::archive_receipt_suppresses_and_idempotently_cleans_a_restored_inbox_copy`; `tests/message.rs` | -| **Durable native delivery ownership** | Each native harness keeps one monotone evidence chain per inbox filename in `delivery-ledger.json`. Ownership is persisted before the transport call, so a crash between attempt and evidence reopens as an ambiguous attempt that holds instead of re-sending. Evidence only advances, and only to a phase the harness can actually observe: Codex proves transport acceptance and consumption, OpenCode transport acceptance and durable read-back, and a record claiming any other phase is refused. Only an authoritative negative receipt re-authorizes a retry. Settlement stays with the recipient: an entry is released only on a phase its profile releases, and is otherwise reconciled away when the message leaves the unread set. A foreign, malformed, or self-inconsistent record quarantines delivery without rewriting it, and the persisted ledger is owner-only with no temp residue. | `src/delivery_ledger.rs::tests::begin_persists_attempted_before_transport`; `src/delivery_ledger.rs::tests::positive_evidence_never_downgrades`; `src/delivery_ledger.rs::tests::profiles_accept_only_evidence_their_harness_can_produce`; `src/delivery_ledger.rs::tests::negative_receipt_is_the_only_retry_authority`; `src/delivery_ledger.rs::tests::opencode_persistence_holds_until_archive`; `src/delivery_ledger.rs::tests::foreign_or_malformed_state_quarantines_without_rewriting`; `src/delivery_ledger.rs::tests::every_entry_must_validate_its_own_correlation`; `src/delivery_ledger.rs::tests::the_persisted_ledger_is_owner_only_and_leaves_no_temp_residue` | +| **Durable native delivery ownership** | Each native harness keeps one monotone evidence chain per inbox filename in `delivery-ledger.json`. Ownership is persisted before the transport call, so a crash between attempt and evidence reopens as an ambiguous attempt that holds instead of re-sending. Evidence only advances, and only to a phase the harness can actually observe: Codex proves transport acceptance and consumption, OpenCode transport acceptance and durable read-back, and a record claiming any other phase is refused. Only an authoritative negative receipt re-authorizes a retry. Settlement stays with the recipient: an entry is released only on a phase its profile releases, and is otherwise reconciled away when the message leaves the unread set. A foreign, malformed, or self-inconsistent record quarantines delivery without rewriting it, and the persisted ledger is owner-only with no temp residue. A delivery attempt an earlier release left behind is carried forward rather than re-sent: the pre-ledger `delivery-state.json` record is translated exactly once, when no ledger file exists, into an asserted phase that holds the delivery and authorizes no transport until this build observes evidence of its own. | `src/delivery_ledger.rs::tests::begin_persists_attempted_before_transport`; `src/delivery_ledger.rs::tests::positive_evidence_never_downgrades`; `src/delivery_ledger.rs::tests::profiles_accept_only_evidence_their_harness_can_produce`; `src/delivery_ledger.rs::tests::negative_receipt_is_the_only_retry_authority`; `src/delivery_ledger.rs::tests::opencode_persistence_holds_until_archive`; `src/delivery_ledger.rs::tests::foreign_or_malformed_state_quarantines_without_rewriting`; `src/delivery_ledger.rs::tests::every_entry_must_validate_its_own_correlation`; `src/delivery_ledger.rs::tests::the_persisted_ledger_is_owner_only_and_leaves_no_temp_residue`; `src/delivery_ledger.rs::tests::an_asserted_phase_suppresses_a_duplicate_and_authorizes_no_transport`; `src/migrations/delivery_state/mod.rs::tests::a_carried_forward_attempt_is_an_assertion_that_authorizes_no_transport`; `src/migrations/delivery_state/mod.rs::tests::translation_happens_once_and_leaves_the_old_record_in_place`; `src/migrations/delivery_state/mod.rs::tests::deletion_trigger_absent_old_record_makes_this_module_a_no_op` | | **Idempotent service requests** | A declared non-agent service principal publishes one exact JSON request per caller-supplied idempotency key to a canonical Agent Spec inbox. Concurrent or crash-replayed publication reuses the reserved filename; conflicting key reuse fails. The typed reply routes to the principal's canonical inbox without an Agent Spec identity or orphan mailbox. | `tests/request_cli.rs::stable_request_key_atomically_deduplicates_one_canonical_agent_message`; `tests/request_cli.rs::concurrent_replays_publish_exactly_one_request`; `tests/request_cli.rs::typed_reply_routes_to_the_principal_and_status_is_a_tagged_json_union`; `tests/request_cli.rs::request_api_rejects_agent_impersonation_and_unknown_flat_principals` | | **Bounded idempotent stream ingress** | A running agent accepts events only for a declared stream. Within the retained 128-receipt ring, concurrent or crash-replayed `(stream, event-id)` publication reuses one canonical filename, conflicting content fails, and supersession archives only the matching keyed predecessor or the stream-wide head through ordinary archive semantics. State remains bounded and honestly treats an identity evicted from the ring as new without searching inbox or archive history. Events do not write the Sent ledger and DING marks them as stream work. | `tests/event_e2e.rs::stable_event_identity_publishes_exactly_one_canonical_message`; `tests/event_e2e.rs::concurrent_replays_publish_exactly_one_event`; `tests/event_e2e.rs::conflicting_reuse_and_undeclared_or_suspended_ingress_fail_closed`; `tests/event_e2e.rs::supersede_collapses_only_the_matching_key_and_preserves_archive_receipts`; `tests/event_e2e.rs::keyless_supersede_replaces_the_stream_wide_head`; `tests/event_e2e.rs::crash_replay_honors_an_archive_receipt_and_never_restores_the_inbox_copy`; `tests/event_e2e.rs::subject_frontmatter_injection_is_refused_before_any_write`; `tests/event_e2e.rs::stream_state_is_bounded_and_forgets_only_beyond_its_honest_horizon`; `tests/event_e2e.rs::event_emit_cli_returns_a_stable_json_receipt_and_ding_marks_the_record`; `tests/stream_authoring_cli.rs::a_direct_adapter_launch_executes_the_exact_event_cli_contract`; `tests/run.rs::suspend_and_resume_relaunch_the_agent_and_stream_together` | | **Fail-closed observed native DING** | Each unread message becomes one normalized `[DING]` frame. Fresh delivery records ownership, then preserves the one combined bracketed-paste, 0.5 second delay, and Return transaction. PTY and Return success are transport only: `Delivered` additionally requires adapter classification of the expected notice text in a submitted-prompt or queued-message pattern while the lowest live composer is empty or an accepted idle placeholder. Retry never re-pastes and may send one bare Return only after two adjacent `RetainedSafe` observations. A maintained adapter's positive `NotRetained` observation releases only an already archived staged head; unread, blocked, timed-out, errored, unknown, and unrecognized states retain ownership and later FIFO work remains blocked. Ownership prevents duplicate paste across command failures, receipt ambiguity, archive races, and restart adoption without letting a vanished archived head block FIFO indefinitely. Startup backlog otherwise becomes one generic recovery DING; new arrivals remain FIFO; `busy` delivers immediately; only fresh `dnd` defers. | `src/ding/mod.rs::poke_text_normalizes_and_bounds_untrusted_fields`; `src/ding/mod.rs::malicious_controls_cannot_escape_the_single_paste_frame`; `src/ding/mod.rs::pty_delivery_uses_face607_delay_order_and_seconds`; `src/ding/mod.rs::maintained_composer_classifiers_require_exact_idle_state`; `src/ding/mod.rs::successful_transport_with_retained_or_unproven_pixels_is_not_delivered`; `src/ding/mod.rs::ambiguous_transport_receipt_and_retry_errors_retain_staged_ownership`; `src/ding/mod.rs::adapter_recognized_notice_with_an_empty_live_composer_is_a_positive_receipt`; `src/ding/mod.rs::staged_retry_submits_only_retained_safe_and_requires_a_receipt`; `src/ding/mod.rs::staged_retry_keeps_unproven_and_retained_blocked_owned`; `src/ding/mod.rs::staged_ownership_survives_archive_and_never_repastes`; `src/ding/mod.rs::archived_not_retained_releases_fifo_without_repasting_owned_notice`; `src/ding/mod.rs::unread_not_retained_keeps_fifo_ownership_without_repasting`; `src/ding/mod.rs::pty_commands_have_a_real_outer_timeout`; `src/ding/mod.rs::session_watch_has_startup_grace_debounce_and_live_reset`; `src/ding/mod.rs::new_arrivals_is_fifo_and_archive_receipts_prevent_reding`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry`; `src/ding/mod.rs::startup_recovery_notice_retries_in_memory`; `src/ding/mod.rs::startup_backlog_gets_one_generic_recovery_then_new_arrivals_poke` | diff --git a/docs/vrs/.delta/DELTA-006-delivery-state-v1-arm.md b/docs/vrs/.delta/DELTA-006-delivery-state-v1-arm.md new file mode 100644 index 00000000..938d8ff0 --- /dev/null +++ b/docs/vrs/.delta/DELTA-006-delivery-state-v1-arm.md @@ -0,0 +1,85 @@ +# DELTA-006: the pre-ledger `delivery-state.json` boundary arm outlives its own necessity + +Status: open + +## Divergence + +The canonical delivery ledger owns exactly one schema, `st2.delivery-ledger.v1`, +and `src/delivery_ledger.rs` is written as though only that format had ever +shipped. It nevertheless carries one statement that knows otherwise: the +`ErrorKind::NotFound` arm of `Ledger::open` calls +`crate::migrations::delivery_state::recover`, which reads the single-binding +`delivery-state.json` record every pre-ledger release wrote and carries an +in-flight attempt forward as an assertion instead of letting it be re-sent. + +That arm is correct today and is dead weight the moment no such record exists +anywhere. Nothing in the code can observe that condition, so the divergence is +between an implementation that must still translate a retired format and a +design that owns one format — and it closes by deletion, on a fleet +observation, not by an amendment. + +## VRS + +No ratified requirement mentions either record. The rule the arm exists to +preserve is [`DING-R07`](../01-ding/requirements.md) staged ownership: once an +attempt has started, ambiguity about whether it landed is resolved by +inspection, never by pasting the same notice again. A release boundary is +exactly such an ambiguity — the old binary's record is the only evidence that +an attempt was made — so dropping that record on upgrade would resolve the +ambiguity by re-sending, which DING-R07 forbids and DING-T01 answers the other +way: every ambiguous case resolves toward deferral. + +Requirements therefore need no change. What needs recording is that a piece of +the implementation is deliberately temporary, with the observation that ends +it. + +## Implementation + +`src/migrations/delivery_state/` owns the whole boundary: `mod.rs` (the entry +point, the ownership filter, the legacy filename), `codex_v1.rs`, and +`opencode_v1.rs` (one retired wire struct each, plus the meaning of its +labels — Codex's `accepted` was a typed in-turn receipt and grades to +`consumed`, OpenCode's was a storage read-back and grades to `persisted`). + +Canonical code gains one version-free concept, `Attestation{Observed, +Asserted}` on `Entry`: a phase this build graded versus a phase another +authority asserted. An assertion bounds what already happened, so it suppresses +a duplicate; it is not an observation, so it authorizes no transport until this +build sees something itself. That distinction is permanent and would be needed +by any future asserting authority, so it stays when the arm goes. + +Deletion is `git rm -r src/migrations` plus replacing the seam arm with +`Ok(())`, which is byte-for-byte a first run on a fresh seat. Measured cost of +that deletion: one compile error, at the seam. + +## Direction + +update implementation + +## Resolution Signal + +Both commands below print nothing, on every admitted host, for seven +consecutive days: + +```sh +state="${XDG_STATE_HOME:-$HOME/.local/state}/st2" + +# 1. No pre-ledger record is left beside any per-harness state dir, so no +# unread attempt can still need carrying forward. +find "$state/codex" "$state/opencode" -maxdepth 2 -name delivery-state.json -print + +# 2. No ledger entry still holds a phase this fleet never observed. While one +# exists, the translation that produced it is load-bearing. +find "$state/codex" "$state/opencode" -maxdepth 2 -name delivery-ledger.json -print0 \ + | xargs -0 -r jq -r 'select([.entries[].attestation] | any(. == "asserted")) | input_filename' +``` + +Clause 1 also requires that rollback to a pre-ledger release has stopped being +supported: while it is supported, a rolled-back binary can write a new +`delivery-state.json`, and the roll-forward window it opens is pinned by +`migrations::delivery_state::tests::a_rollback_then_roll_forward_does_not_see_the_record_written_in_between`. + +The local half of the trigger — that with no old record present the module +contributes nothing and writes nothing, so removing it cannot change observed +behaviour — is asserted by +`migrations::delivery_state::tests::deletion_trigger_absent_old_record_makes_this_module_a_no_op`. diff --git a/src/delivery_ledger.rs b/src/delivery_ledger.rs index 9e8f8971..2ca3cb24 100644 --- a/src/delivery_ledger.rs +++ b/src/delivery_ledger.rs @@ -4,6 +4,12 @@ //! harness-specific observations into [`Evidence`]; this module owns persistence, phase grading, //! retry authorization, and binding isolation. Inbox archive remains the recipient's settlement //! authority and is reconciled through [`Ledger::prune`]. +//! +//! One phase can reach this ledger without this build observing anything: an attempt an earlier +//! release made and left behind. [`Attestation`] is the whole vocabulary for that — an asserted +//! phase bounds what already happened, so it suppresses a duplicate, and it is not evidence, so +//! it authorizes no transport. The translation itself lives outside this module, behind the one +//! seam in [`Ledger::open`]. use std::fs; use std::io::Write as _; @@ -133,6 +139,21 @@ pub enum Evidence { Consumed, } +/// Whether this build observed the evidence behind a phase, or another authority asserted it. +/// +/// An assertion is a true lower bound on what already happened, so it may suppress a duplicate; +/// it is not an observation, so it authorizes no transport until fresh evidence arrives. Nothing +/// here names the authority: any party that can bound an attempt this build never watched +/// asserts, and the record boundary in `crate::migrations::delivery_state` is one such party. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum Attestation { + /// This build graded the evidence that set the phase through [`Profile::graded`]. + Observed, + /// Another authority asserted the phase: enough to hold a delivery, never enough to send one. + Asserted, +} + #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub struct Entry { @@ -140,6 +161,8 @@ pub struct Entry { pub binding: String, pub correlation: Correlation, pub phase: Phase, + /// Whether `phase` is this build's own grading or a claim it accepted from elsewhere. + pub attestation: Attestation, /// The runtime incarnation that made the attempt. Live evidence acknowledges only its own /// incarnation; history reconciliation settles attempts from earlier incarnations. #[serde(default, skip_serializing_if = "Option::is_none")] @@ -169,6 +192,9 @@ pub enum HoldReason { AmbiguousAttempt, UnreadReceipt, NegativeReceipt, + /// The phase was asserted, not observed: enough to suppress a duplicate, never enough to + /// authorize a transport. Only fresh evidence about the world clears it. + UnattestedClaim, Quarantined, Settled, } @@ -222,7 +248,24 @@ impl Ledger { }; let outcome = match fs::read(path) { Ok(bytes) => ledger.load(&bytes, &correlate), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + // THE RECOVERY SEAM. The only statement in this module that knows any other delivery + // record has ever existed; deleting `crate::migrations::delivery_state` deletes this + // arm and nothing else. What it degrades to — `Ok(())`, no entries — is exactly a + // first run on a fresh seat. DELETION TRIGGER: docs/vrs/.delta/DELTA-006. + Err(error) if error.kind() == std::io::ErrorKind::NotFound => path + .parent() + .map_or_else( + || Ok(Vec::new()), + |state_dir| { + crate::migrations::delivery_state::recover( + state_dir, + profile.harness, + agent, + &correlate, + ) + }, + ) + .and_then(|recovered| ledger.seed(recovered, &correlate)), Err(error) => Err(error).with_context(|| format!("reading delivery ledger {}", path.display())), }; if let Err(error) = outcome { @@ -249,18 +292,46 @@ impl Ledger { record.agent == self.record.agent, "delivery ledger belongs to a different agent" ); - for entry in &record.entries { + self.accept(&record.entries, correlate)?; + record.runtime_id.clone_from(&self.record.runtime_id); + self.record = record; + Ok(()) + } + + /// Accept entries this process did not itself create, then make them durable. + /// + /// They are validated exactly as bytes from this module's own file would be, so a claim about + /// evidence the harness cannot produce fails closed instead of being written. Recovery + /// happens once: the ledger file's existence is what stops it happening twice, and a driver + /// that never delivered leaves no file behind. + fn seed( + &mut self, + entries: Vec, + correlate: &impl Fn(&str, &str) -> String, + ) -> Result<()> { + if entries.is_empty() { + return Ok(()); + } + self.accept(&entries, correlate)?; + self.record.entries = entries; + self.persist() + } + + /// Every check a set of entries from outside this process must pass. + fn accept( + &self, + entries: &[Entry], + correlate: &impl Fn(&str, &str) -> String, + ) -> Result<()> { + for entry in entries { self.validate(entry, correlate)?; } anyhow::ensure!( - record - .entries + entries .windows(2) .all(|pair| pair[0].binding == pair[1].binding), "delivery ledger holds entries from more than one binding" ); - record.runtime_id.clone_from(&self.record.runtime_id); - self.record = record; Ok(()) } @@ -335,6 +406,9 @@ impl Ledger { entry.correlation = begin.correlation; entry.incarnation = begin.incarnation; entry.phase = entry.phase.max(Phase::Attempted); + // This build is about to transport it, so the attempt is no longer someone + // else's claim about the world. + entry.attestation = Attestation::Observed; entry.negative = None; entry.clone() } @@ -344,6 +418,7 @@ impl Ledger { binding: begin.binding, correlation: begin.correlation, phase: Phase::Attempted, + attestation: Attestation::Observed, incarnation: begin.incarnation, negative: None, }; @@ -371,6 +446,8 @@ impl Ledger { return Ok(Some(entry.phase)); } entry.phase = phase; + // This build graded the evidence, so the phase is no longer a carried-forward claim. + entry.attestation = Attestation::Observed; entry.negative = None; self.persist()?; Ok(Some(phase)) @@ -395,6 +472,9 @@ impl Ledger { return Ok(Retention::Hold(HoldReason::NegativeReceipt)); } entry.negative = Some(receipt); + // An authoritative absence is itself an observation about this attempt, and it is the + // only receipt that may re-authorize a transport of a carried-forward one. + entry.attestation = Attestation::Observed; self.persist()?; Ok(Retention::Hold(HoldReason::NegativeReceipt)) } @@ -415,6 +495,9 @@ impl Ledger { if entry.phase >= Phase::Persisted { return Retention::Hold(HoldReason::UnreadReceipt); } + if entry.attestation == Attestation::Asserted { + return Retention::Hold(HoldReason::UnattestedClaim); + } Retention::Hold(HoldReason::AmbiguousAttempt) } @@ -475,6 +558,30 @@ impl Ledger { } } +/// Build an entry whose phase another authority asserted rather than this build observing it. +/// +/// The only constructor the recovery seam may use. The phase it carries is still checked against +/// the harness [`Profile`] by [`Ledger::seed`], so a claim of evidence the harness cannot produce +/// fails closed instead of being written, and [`Attestation::Asserted`] keeps the entry holding +/// until this build observes something: it can suppress a duplicate, it can authorize nothing. +pub fn asserted( + filename: String, + binding: String, + correlation: Correlation, + phase: Phase, + incarnation: Option, +) -> Entry { + Entry { + filename, + binding, + correlation, + phase, + attestation: Attestation::Asserted, + incarnation, + negative: None, + } +} + /// Durable replacement: file bytes reach disk before rename, then the directory entry is synced. /// /// The temp file is created exclusively at `0600` under a name unique to this process and write, @@ -578,6 +685,65 @@ mod tests { ); } + /// The safety property the whole record boundary rests on: an asserted phase is a bound on + /// what already happened, so it holds the delivery, and it is not evidence, so it authorizes + /// no transport. This build's own observation is what clears it. + #[test] + fn an_asserted_phase_suppresses_a_duplicate_and_authorizes_no_transport() { + let tmp = tempfile::tempdir().unwrap(); + let mut ledger = open(tmp.path(), Harness::Codex); + ledger + .seed( + vec![asserted( + FILE_A.to_owned(), + "thread-main".to_owned(), + Correlation::native(correlation("thread-main", FILE_A)), + Phase::Attempted, + Some("incarnation-0".to_owned()), + )], + &correlation, + ) + .unwrap(); + assert_eq!( + ledger.retention(FILE_A), + Retention::Hold(HoldReason::UnattestedClaim) + ); + assert_eq!( + ledger.retry(FILE_A), + RetryDecision::Hold(HoldReason::UnattestedClaim) + ); + + // The seed is durable, so a restart still holds instead of re-sending. + let reopened = open(tmp.path(), Harness::Codex); + assert_eq!( + reopened.entry(FILE_A).unwrap().attestation, + Attestation::Asserted + ); + assert_eq!( + reopened.retry(FILE_A), + RetryDecision::Hold(HoldReason::UnattestedClaim) + ); + + // An authoritative absence is an observation: it clears the claim and re-authorizes one + // transport of the same identity. + let mut ledger = open(tmp.path(), Harness::Codex); + ledger.negative(FILE_A, NegativeReceipt::Absent).unwrap(); + assert_eq!( + ledger.entry(FILE_A).unwrap().attestation, + Attestation::Observed + ); + assert_eq!(ledger.retry(FILE_A), RetryDecision::Retry); + + // And so is a graded receipt: the entry stops being a carried-forward claim. + let mut ledger = open(tmp.path(), Harness::Codex); + ledger.record(FILE_A, Evidence::Consumed).unwrap(); + assert_eq!( + ledger.entry(FILE_A).unwrap().attestation, + Attestation::Observed + ); + assert_eq!(ledger.retention(FILE_A), Retention::Release); + } + #[test] fn the_persisted_ledger_is_owner_only_and_leaves_no_temp_residue() { use std::os::unix::fs::PermissionsExt as _; @@ -709,6 +875,7 @@ mod tests { binding: "thread-main".to_owned(), correlation: Correlation::native("injected"), phase: Phase::Attempted, + attestation: Attestation::Observed, incarnation: None, negative: None, }], diff --git a/src/lib.rs b/src/lib.rs index e462bf49..ccd81be0 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -38,6 +38,7 @@ pub mod materialize; pub mod message; pub mod omp_session; pub mod metrics; +pub mod migrations; pub mod opencode_session; pub mod park; pub mod pi_channel; diff --git a/src/migrations/delivery_state/codex_v1.rs b/src/migrations/delivery_state/codex_v1.rs new file mode 100644 index 00000000..0883f0ac --- /dev/null +++ b/src/migrations/delivery_state/codex_v1.rs @@ -0,0 +1,82 @@ +//! `st2.codex-delivery-state.v1` — the Codex driver's own delivery record, written by every +//! release before `st2.delivery-ledger.v1`. +//! +//! Its loader `ensure!`d its schema and denied unknown fields, so it refuses to start on anything +//! it does not recognize. That is why the canonical record is a new filename rather than a bump of +//! this one, and why nothing here writes: this file only reads bytes an older binary left. + +use anyhow::Result; +use serde::Deserialize; + +use super::{Adopting, Version}; +use crate::delivery_ledger::{self, Correlation, Entry, Phase}; +use crate::message; + +pub(super) const SCHEMA: &str = "st2.codex-delivery-state.v1"; + +/// The v1 wire struct. `runtimeId` is present in the bytes and deliberately not modelled: v1 +/// compared it and hard-errored on drift, and this translation carries a drifted record forward +/// instead (see [`super::translate`]). +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub(super) struct Record { + schema: String, + agent: String, + #[serde(default)] + runtime_incarnation: String, + thread_id: String, + filename: String, + client_id: String, + phase: Label, +} + +/// v1's two-value phase. `Accepted` was written **only** from the typed +/// `item/completed{userMessage, clientId}` event inside a turn, so it means the model received +/// the message — Codex's true ceiling. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)] +#[serde(rename_all = "camelCase")] +enum Label { + Attempted, + Accepted, +} + +impl Version for Record { + const SCHEMA: &'static str = SCHEMA; + + fn schema(&self) -> &str { + &self.schema + } + + fn agent(&self) -> &str { + &self.agent + } +} + +impl TryFrom> for Entry { + type Error = anyhow::Error; + + fn try_from(adopting: Adopting<'_, Record>) -> Result { + let Adopting { record, correlate } = adopting; + anyhow::ensure!( + message::is_message_filename(&record.filename) && !record.thread_id.is_empty(), + "v1 Codex delivery state has an invalid binding or filename" + ); + anyhow::ensure!( + record.client_id == correlate(&record.thread_id, &record.filename), + "v1 Codex delivery state correlation does not match its binding" + ); + let phase = match record.phase { + Label::Attempted => Phase::Attempted, + Label::Accepted => Phase::Consumed, + }; + Ok(delivery_ledger::asserted( + record.filename, + record.thread_id, + Correlation::native(record.client_id), + phase, + // The typed receipt is a live frame, so an attempt may only be acknowledged by the + // incarnation that made it; keeping the old one is what forces a resume sweep. + Some(record.runtime_incarnation).filter(|value| !value.is_empty()), + )) + } +} diff --git a/src/migrations/delivery_state/mod.rs b/src/migrations/delivery_state/mod.rs new file mode 100644 index 00000000..88df6bc2 --- /dev/null +++ b/src/migrations/delivery_state/mod.rs @@ -0,0 +1,483 @@ +//! Translation of the per-driver `delivery-state.json` records into +//! [`crate::delivery_ledger::Entry`]. +//! +//! Before `st2.delivery-ledger.v1`, each native transport kept its own single-binding record at +//! `/delivery-state.json` holding one `{binding, filename, correlation, phase in +//! {Attempted, Accepted}}`. The two records had the same field *shape* and different meanings: +//! Codex wrote `Accepted` only from the typed `item/completed{userMessage, clientId}` inside a +//! turn (the model received it), OpenCode from `GET /session/{s}/message/{m}` returning 200 (the +//! server stored it). Nothing on disk said which. That difference — and the field spellings, and +//! the filename, and the schema strings — lives here and nowhere else. +//! +//! One file per legacy version: [`codex_v1`], [`opencode_v1`]. Each owns its wire struct, the +//! meaning of its labels, and a `TryFrom> for Entry`. A future +//! `delivery-state.v2` would be one more file plus one match arm in [`recover`]. +//! +//! Two properties make this safe to translate rather than migrate in place: +//! +//! * **Fresh namespace.** The canonical record is a different filename, so nothing is rewritten +//! and a translation can be re-derived from bytes that are still there. +//! * **Assertion, not observation.** Every entry this module produces is +//! [`crate::delivery_ledger::Attestation::Asserted`] via +//! [`crate::delivery_ledger::asserted`], so a carried-forward phase can suppress a duplicate +//! and can never authorize a transport. That is the whole safety argument, and it is enforced +//! by the canonical validator, not by care taken here. +//! +//! # Deletion trigger +//! +//! `docs/vrs/.delta/DELTA-006-delivery-state-v1-arm.md`, whose Resolution Signal is the live +//! query that makes this directory removable — no `delivery-state.json` beside a ledger on any +//! admitted host for seven days, and no ledger entry still carrying an asserted phase. Deleting +//! it is this directory plus the one seam statement in +//! [`crate::delivery_ledger::Ledger::open`]. The local half of the trigger is +//! [`tests::deletion_trigger_absent_old_record_makes_this_module_a_no_op`]. + +mod codex_v1; +mod opencode_v1; + +use anyhow::{Context, Result}; +use serde::de::DeserializeOwned; +use std::fs; +use std::path::Path; + +use crate::delivery_ledger::{Entry, Harness}; + +/// The filename every pre-ledger release wrote. Named here only. +const LEGACY_FILE: &str = "delivery-state.json"; + +/// A legacy record together with the canonical correlation derivation it has to re-prove. +/// +/// The derivation is the same function the transport uses, so a record whose correlation does not +/// match its own binding and filename is provably not this agent's. `TryFrom` cannot carry that +/// context, so it rides in the wrapper. +pub(super) struct Adopting<'a, T> { + pub record: T, + pub correlate: &'a dyn Fn(&str, &str) -> String, +} + +/// One legacy delivery-state version. +pub(super) trait Version: DeserializeOwned { + /// The exact `schema` string the release that wrote this record stamped on it. + const SCHEMA: &'static str; + fn schema(&self) -> &str; + fn agent(&self) -> &str; +} + +/// Translate the legacy record in `state_dir`, if any, into canonical entries. +/// +/// The canonical caller invokes this exactly once — when no ledger file exists — and makes the +/// result durable, so the ledger file's existence is what stops the translation happening twice. +/// Called with the ledger absent and no legacy record present it returns no entries, which is +/// byte-for-byte the behaviour of this module not existing. +pub(crate) fn recover( + state_dir: &Path, + harness: Harness, + agent: &str, + correlate: &F, +) -> Result> +where + F: Fn(&str, &str) -> String, +{ + let path = state_dir.join(LEGACY_FILE); + let bytes = match fs::read(&path) { + Ok(bytes) => bytes, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(error) => { + return Err(error) + .with_context(|| format!("reading legacy delivery state {}", path.display())); + } + }; + let correlate: &dyn Fn(&str, &str) -> String = correlate; + match harness { + Harness::Codex => translate::(&bytes, agent, correlate), + Harness::OpenCode => translate::(&bytes, agent, correlate), + } +} + +/// Decode, filter by ownership, convert. +/// +/// A record naming another schema, another harness's field set, or another agent is *ignored*: +/// it is not this ledger's authority, and refusing to start over someone else's file would +/// deliver nothing at all. Unreadable bytes are ignored for the same reason. But a record that is +/// ours and contradicts itself fails closed, because the canonical caller turns that error into a +/// quarantine and a quarantine is exactly the right answer to "I hold a delivery record I cannot +/// interpret". +/// +/// The runtime id is deliberately not compared, for a different reason on each harness. OpenCode's +/// loader never looked at it, so comparing it would drop a record the old binary would have acted +/// on. Codex's loader did compare it — and hard-errored, refusing to start — but a drifted record +/// still describes a real attempt this recipient made, so it is carried forward as an assertion. +/// Held that way, a resume sweep can settle or refuse it BEFORE anything is sent, which is +/// strictly better than ignoring it and opening a second delivery for the same message. +fn translate( + bytes: &[u8], + agent: &str, + correlate: &dyn Fn(&str, &str) -> String, +) -> Result> +where + V: Version, + for<'a> Entry: TryFrom, Error = anyhow::Error>, +{ + let Ok(record) = serde_json::from_slice::(bytes) else { + return Ok(Vec::new()); + }; + if record.schema() != V::SCHEMA || record.agent() != agent { + return Ok(Vec::new()); + } + Ok(vec![Entry::try_from(Adopting { record, correlate })?]) +} + +/// Legacy records in the exact shape a pre-ledger release wrote them, for tests that need to +/// stand at the migration boundary. +/// +/// A test reaches for these instead of spelling the old field names itself, so the wire shape of +/// a retired format is authored in one place and no other test module has to know it existed. +#[cfg(test)] +pub(crate) mod fixture { + use serde_json::{Value, json}; + use std::path::Path; + + pub(crate) fn codex_attempted( + agent: &str, + runtime_id: &str, + incarnation: &str, + thread_id: &str, + filename: &str, + client_id: &str, + ) -> Value { + json!({ + "schema": super::codex_v1::SCHEMA, + "agent": agent, + "runtimeId": runtime_id, + "runtimeIncarnation": incarnation, + "threadId": thread_id, + "filename": filename, + "clientId": client_id, + "phase": "attempted", + }) + } + + pub(crate) fn opencode_attempted( + agent: &str, + runtime_id: &str, + session_id: &str, + filename: &str, + message_id: &str, + ) -> Value { + json!({ + "schema": super::opencode_v1::SCHEMA, + "agent": agent, + "runtimeId": runtime_id, + "sessionId": session_id, + "filename": filename, + "messageId": message_id, + "phase": "attempted", + }) + } + + /// Write `record` where a pre-ledger release would have left it. + pub(crate) fn place(state_dir: &Path, record: &Value) { + std::fs::create_dir_all(state_dir).unwrap(); + std::fs::write( + state_dir.join(super::LEGACY_FILE), + serde_json::to_vec(record).unwrap(), + ) + .unwrap(); + } + + pub(crate) fn path(state_dir: &Path) -> std::path::PathBuf { + state_dir.join(super::LEGACY_FILE) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::delivery_ledger::{ + Attestation, HoldReason, LEDGER_FILE, LEDGER_SCHEMA, Ledger, Phase, Retention, + RetryDecision, + }; + + const FILE_A: &str = "1786380000000-aaa111.md"; + + fn correlate(binding: &str, filename: &str) -> String { + format!("corr:{binding}:{filename}") + } + + fn open(state_dir: &Path, harness: Harness) -> Ledger { + Ledger::open( + &state_dir.join(LEDGER_FILE), + harness.profile(), + "h.worker", + "h.worker", + correlate, + ) + } + + /// The local half of the deletion trigger: with no legacy record present this module produces + /// nothing and causes no write, so removing it cannot change observable behaviour. The fleet + /// half is the live query in the delta record named in the module doc. + #[test] + fn deletion_trigger_absent_old_record_makes_this_module_a_no_op() { + for harness in [Harness::Codex, Harness::OpenCode] { + let tmp = tempfile::tempdir().unwrap(); + let state_dir = tmp.path().join("state"); + assert!( + recover(&state_dir, harness, "h.worker", &correlate) + .unwrap() + .is_empty() + ); + let ledger = open(&state_dir, harness); + assert!(ledger.quarantined().is_none()); + assert!(ledger.entries().is_empty()); + assert!( + !state_dir.exists(), + "a driver that never delivered leaves no record behind" + ); + } + } + + #[test] + fn accepted_adopts_at_the_evidence_each_harness_actually_proved() { + // Codex wrote `Accepted` only from the typed completed user message: consumption, its + // true ceiling, so the delivery is settled and never re-offered. + let codex = tempfile::tempdir().unwrap(); + let codex_dir = codex.path().join("state"); + let mut record = fixture::codex_attempted( + "h.worker", + "h.worker", + "incarnation-1", + "thread-main", + FILE_A, + &correlate("thread-main", FILE_A), + ); + record["phase"] = "accepted".into(); + fixture::place(&codex_dir, &record); + let ledger = open(&codex_dir, Harness::Codex); + assert_eq!(ledger.entry(FILE_A).unwrap().phase, Phase::Consumed); + assert_eq!(ledger.retention(FILE_A), Retention::Release); + + // OpenCode wrote it from a storage read-back. Mapping that to consumption would make the + // stored-but-never-admitted class permanently unretryable, so it adopts as `persisted` + // and holds. + let opencode = tempfile::tempdir().unwrap(); + let opencode_dir = opencode.path().join("state"); + let mut record = fixture::opencode_attempted( + "h.worker", + "h.worker", + "ses_target", + FILE_A, + &correlate("ses_target", FILE_A), + ); + record["phase"] = "accepted".into(); + fixture::place(&opencode_dir, &record); + let ledger = open(&opencode_dir, Harness::OpenCode); + assert_eq!(ledger.entry(FILE_A).unwrap().phase, Phase::Persisted); + assert_eq!( + ledger.retention(FILE_A), + Retention::Hold(HoldReason::UnreadReceipt) + ); + } + + #[test] + fn a_carried_forward_attempt_is_an_assertion_that_authorizes_no_transport() { + let tmp = tempfile::tempdir().unwrap(); + let state_dir = tmp.path().join("state"); + fixture::place( + &state_dir, + &fixture::codex_attempted( + "h.worker", + "h.other-runtime", + "incarnation-0", + "thread-main", + FILE_A, + &correlate("thread-main", FILE_A), + ), + ); + let ledger = open(&state_dir, Harness::Codex); + let entry = ledger.entry(FILE_A).unwrap(); + assert_eq!(entry.attestation, Attestation::Asserted); + assert_eq!(entry.phase, Phase::Attempted); + assert_eq!( + entry.incarnation.as_deref(), + Some("incarnation-0"), + "the attempt keeps the incarnation that made it, so no live frame can settle it" + ); + assert_eq!( + ledger.retry(FILE_A), + RetryDecision::Hold(HoldReason::UnattestedClaim), + "a drifted runtime id is carried forward and held, not ignored into a second delivery" + ); + } + + #[test] + fn translation_happens_once_and_leaves_the_old_record_in_place() { + let tmp = tempfile::tempdir().unwrap(); + let state_dir = tmp.path().join("state"); + fixture::place( + &state_dir, + &fixture::opencode_attempted( + "h.worker", + "h.worker", + "ses_target", + FILE_A, + &correlate("ses_target", FILE_A), + ), + ); + let mut ledger = open(&state_dir, Harness::OpenCode); + assert_eq!(ledger.entries().len(), 1); + // The recipient archives the message; ownership is released and the entry is gone. + ledger.prune(|_| false).unwrap(); + + // The legacy record is still on disk, and it is NOT translated a second time: the ledger + // file's existence is the once-only fence. + assert!(fixture::path(&state_dir).is_file()); + let reopened = open(&state_dir, Harness::OpenCode); + assert!(reopened.quarantined().is_none()); + assert!( + reopened.entries().is_empty(), + "a settled delivery is not resurrected by the record it was translated from" + ); + } + + /// The once-only fence's known limit, pinned so that changing it is a deliberate edit. + /// + /// The fence is "no ledger file exists". After a rollback to a pre-ledger release and a roll + /// FORWARD again, the ledger file is already there and the record the old binary wrote in + /// between is never read: the new binary starts with no entry for that filename and its first + /// transport is not a retry. Rollback to a pre-ledger release is out of support, so this + /// window closes by policy rather than by code. + /// + /// Closing it in code costs a second seam: recovery would have to be consultable per filename + /// (canonical would ask "can anything contribute an entry for this file?" before authorizing a + /// first transport) or run on every open with merge semantics — which resurrects an entry the + /// recipient already archived until the next `prune`. Both are one statement each; neither is + /// free, and the choice belongs to whoever supports the rollback. + #[test] + fn a_rollback_then_roll_forward_does_not_see_the_record_written_in_between() { + let tmp = tempfile::tempdir().unwrap(); + let state_dir = tmp.path().join("state"); + + // A ledger exists (this release ran once) and holds nothing outstanding. + fs::create_dir_all(&state_dir).unwrap(); + fs::write( + state_dir.join(LEDGER_FILE), + serde_json::to_vec(&serde_json::json!({ + "schema": LEDGER_SCHEMA, + "harness": "codex", + "agent": "h.worker", + "runtimeId": "h.worker", + "entries": [], + })) + .unwrap(), + ) + .unwrap(); + + // The rolled-back release transported a message and wrote its own record. + fixture::place( + &state_dir, + &fixture::codex_attempted( + "h.worker", + "h.worker", + "incarnation-old", + "thread-main", + FILE_A, + &correlate("thread-main", FILE_A), + ), + ); + + // Rolling forward: the ledger file is present, so the seam does not fire. + let rolled_forward = open(&state_dir, Harness::Codex); + assert!(rolled_forward.quarantined().is_none()); + assert!( + rolled_forward.entry(FILE_A).is_none(), + "known limit: the once-only fence is the ledger file, not per-filename coverage" + ); + assert_eq!( + rolled_forward.retry(FILE_A), + RetryDecision::Retry, + "and so the attempt the old binary made is not held" + ); + } + + #[test] + fn a_record_belonging_to_someone_else_is_ignored_and_ours_that_lies_fails_closed() { + // Another agent's record, and another harness's field set: not our authority. + let other = tempfile::tempdir().unwrap(); + let other_dir = other.path().join("state"); + fixture::place( + &other_dir, + &fixture::codex_attempted( + "h.someone-else", + "h.someone-else", + "incarnation-1", + "thread-main", + FILE_A, + &correlate("thread-main", FILE_A), + ), + ); + assert!(open(&other_dir, Harness::Codex).entries().is_empty()); + + let crossed = tempfile::tempdir().unwrap(); + let crossed_dir = crossed.path().join("state"); + fixture::place( + &crossed_dir, + &fixture::opencode_attempted( + "h.worker", + "h.worker", + "ses_target", + FILE_A, + &correlate("ses_target", FILE_A), + ), + ); + let ledger = open(&crossed_dir, Harness::Codex); + assert!(ledger.quarantined().is_none()); + assert!( + ledger.entries().is_empty(), + "an OpenCode record is not a Codex delivery" + ); + + // Ours, and self-contradicting: the correlation does not re-derive from its own binding. + let tampered = tempfile::tempdir().unwrap(); + let tampered_dir = tampered.path().join("state"); + fixture::place( + &tampered_dir, + &fixture::codex_attempted( + "h.worker", + "h.worker", + "incarnation-1", + "thread-main", + FILE_A, + "forged", + ), + ); + let ledger = open(&tampered_dir, Harness::Codex); + assert!( + ledger + .quarantined() + .is_some_and(|reason| reason.contains("does not match its binding")), + "the refusal names itself: {:?}", + ledger.quarantined() + ); + assert!( + fixture::path(&tampered_dir).is_file(), + "a record we refuse to read is not a record we may destroy" + ); + } + + #[test] + fn unreadable_or_unlabelled_bytes_are_ignored_rather_than_quarantining_a_working_pump() { + for body in [ + b"not json at all".to_vec(), + serde_json::to_vec(&serde_json::json!({"schema": "st2.something-else.v1"})).unwrap(), + ] { + let tmp = tempfile::tempdir().unwrap(); + let state_dir = tmp.path().join("state"); + fs::create_dir_all(&state_dir).unwrap(); + fs::write(state_dir.join(LEGACY_FILE), &body).unwrap(); + let ledger = open(&state_dir, Harness::Codex); + assert!(ledger.quarantined().is_none()); + assert!(ledger.entries().is_empty()); + } + } +} diff --git a/src/migrations/delivery_state/opencode_v1.rs b/src/migrations/delivery_state/opencode_v1.rs new file mode 100644 index 00000000..e7b41de4 --- /dev/null +++ b/src/migrations/delivery_state/opencode_v1.rs @@ -0,0 +1,82 @@ +//! `st2.opencode-delivery-state.v1` — the OpenCode driver's own delivery record, written by every +//! release before `st2.delivery-ledger.v1`. +//! +//! Its loader silently discarded a record it could not validate and then re-POSTed the same +//! message id — which on OpenCode 1.18.19 appends the message's parts a second time. That is why +//! the canonical record is a new filename rather than a bump of this one. + +use anyhow::Result; +use serde::Deserialize; + +use super::{Adopting, Version}; +use crate::delivery_ledger::{self, Correlation, Entry, Phase}; +use crate::message; + +pub(super) const SCHEMA: &str = "st2.opencode-delivery-state.v1"; + +/// The v1 wire struct. `runtimeId` is present in the bytes and deliberately not modelled: v1's own +/// load filter never looked at it, so comparing it here would drop a record the old binary would +/// have acted on. +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub(super) struct Record { + schema: String, + agent: String, + session_id: String, + filename: String, + message_id: String, + phase: Label, +} + +/// v1's two-value phase. `Accepted` was written from `GET /session/{s}/message/{m}` returning 200, +/// which proves the server **stored** the exact client message. It is not scheduling and not +/// consumption. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)] +#[serde(rename_all = "camelCase")] +enum Label { + Attempted, + Accepted, +} + +impl Version for Record { + const SCHEMA: &'static str = SCHEMA; + + fn schema(&self) -> &str { + &self.schema + } + + fn agent(&self) -> &str { + &self.agent + } +} + +impl TryFrom> for Entry { + type Error = anyhow::Error; + + fn try_from(adopting: Adopting<'_, Record>) -> Result { + let Adopting { record, correlate } = adopting; + anyhow::ensure!( + message::is_message_filename(&record.filename) && !record.session_id.is_empty(), + "v1 OpenCode delivery state has an invalid binding or filename" + ); + anyhow::ensure!( + record.message_id == correlate(&record.session_id, &record.filename), + "v1 OpenCode delivery state correlation does not match its binding" + ); + let phase = match record.phase { + Label::Attempted => Phase::Attempted, + // Storage, never consumption: mapping this to a released phase would make the + // stored-but-never-admitted class permanently unretryable. + Label::Accepted => Phase::Persisted, + }; + Ok(delivery_ledger::asserted( + record.filename, + record.session_id, + Correlation::native(record.message_id), + phase, + // v1 carried no incarnation, and the read-back is a durable query rather than a live + // frame, so a pre-crash attempt is reconcilable without one. + None, + )) + } +} diff --git a/src/migrations/mod.rs b/src/migrations/mod.rs new file mode 100644 index 00000000..f9417761 --- /dev/null +++ b/src/migrations/mod.rs @@ -0,0 +1,24 @@ +//! Every translation from a format an earlier release wrote into a canonical type this release +//! owns. +//! +//! The rule this tree exists to enforce: **no module outside `src/migrations/` names a format it +//! does not itself write.** Canonical modules know exactly one schema — their own — and reach a +//! translation through a single, named seam. `src/delivery_ledger.rs`'s seam, for example, is one +//! statement in [`crate::delivery_ledger::Ledger::open`]'s "no ledger file" arm. +//! +//! Consequences, all deliberate: +//! +//! * A canonical module's complexity is what it would be if only this version had ever shipped. +//! * Deleting a boundary is a `git rm` plus the removal of its seam statement; nothing else in +//! the tree references the retired format's field names, filename, or old semantics. +//! * Every translation is one directory with one entry point, so "what still reads old bytes?" +//! is answered by `ls src/migrations/`. +//! +//! Each subdirectory MUST name, in its `mod.rs` doc comment, the delta record under +//! `docs/vrs/.delta/` that carries its deletion trigger, and its seam MUST carry a +//! `DELETION TRIGGER: DELTA-NNN` comment. The delta record's Resolution Signal is a live query, +//! so "may this go?" is answered by running a command rather than by reading an opinion. A +//! `deletion_trigger_*` test asserts the local half: that once no old record exists, the module +//! contributes nothing, so removing it cannot change behaviour. + +pub mod delivery_state;