diff --git a/README.md b/README.md index aa74554c..5714967e 100644 --- a/README.md +++ b/README.md @@ -104,12 +104,35 @@ mkdir -p "$bundle/assets" cp examples/native/agent-codex.kdl "$bundle/agent.kdl" cp ./composed-AGENTS.md "$bundle/assets/AGENTS.md" ${EDITOR:-vi} "$bundle/agent.kdl" -st2 agent publish --catalog "$CATALOG" --bundle "$bundle" --expect-absent --json +input_sha256="$(st2 agent digest --bundle "$bundle")" +st2 agent publish --catalog "$CATALOG" --bundle "$bundle" \ + --input-sha256 "$input_sha256" --expect-absent --json ``` Replace ``, ``, ``, and ``. Include every file referenced by `copy` in the bundle. For a later declaration-only update, publish `agent.kdl` with the current declaration's SHA-256 via `--spec ... --expect-sha256 HEX`; sibling assets and state are preserved. +Bind either operation to the exact captured source with the SHA-256 returned by +`st2 agent digest`. + +To prepare and apply a complete declaration-plane replacement without copying +runtime state or workspaces: + +```sh +st2 catalog snapshot --catalog "$CATALOG" --output ./prepared --json +# Edit/render ./prepared, then retain the rootSha256 from the snapshot receipt. +st2 catalog apply --catalog "$CATALOG" --prepared ./prepared \ + --expect-sha256 --json +``` + +`catalog apply` is policy-free. It rejects state/control content, symlinks, +unprojected workspace facts, catalog-local/default PTY roots, and effective +PTY-root changes. Fresh bootstrap is a separate st2+pty transaction, not an +apply mode. A crash leaves a durable marker and content-addressed stage; +`st2 catalog apply --catalog "$CATALOG" --resume --json` resumes without the +original prepared source. Snapshots own the complete bounded `_templates` +library and empty canonical per-agent `.workspace` directory facts, but never +traverse, hash, copy, or delete workspace content. The compact declaration shape is: @@ -415,7 +438,8 @@ ls, up, down, validate, doctor message, ding, agents, status, context, resource, rename, describe env, pty, shell, pretrust hooks, service, eval -agent publish +agent digest, agent publish +catalog snapshot, catalog apply completions ``` @@ -475,6 +499,9 @@ interviewer reply at-or-after the exact kickoff receipt completes it. Canonical verdict. Without the directive, Agent Spec-shaped files inside a fixture remain inert and compact evals retain their flat bus and completion semantics. -`st2 agent publish --catalog ROOT (--spec FILE | --bundle DIR) (--expect-absent | ---expect-sha256 HEX)` is the sole catalog-declaration writer. The publisher admits the complete -prospective catalog under a compare-and-swap lock before making one atomic change. +`st2 agent publish --catalog ROOT (--spec FILE | --bundle DIR) --input-sha256 HEX +(--expect-absent | --expect-sha256 HEX)` is the single-agent declaration writer. +`st2 catalog apply --catalog ROOT +(--prepared DIR --expect-sha256 ROOT_HEX | --resume)` is the complete +declaration-plane writer. Each admits the complete prospective catalog under a +compare-and-swap lock before making one atomic change. diff --git a/crates/agent-spec/src/discovery.rs b/crates/agent-spec/src/discovery.rs index c0e03897..fe1f2ea7 100644 --- a/crates/agent-spec/src/discovery.rs +++ b/crates/agent-spec/src/discovery.rs @@ -80,6 +80,19 @@ pub fn is_catalog_path(root: &Path, path: &Path) -> bool { return false; } + // Canonical state has a stable address independent of whether a declaration is currently + // present. This keeps orphan state after retirement/removal out of discovery and out of a + // whole-catalog transaction's declaration identity. + if components.first().and_then(|name| name.to_str()) == Some("agents") + && components.len() >= 4 + && matches!( + components[3].to_str(), + Some("resources" | "archive" | "inbox" | "status") + ) + { + return false; + } + let mut parent = root.to_path_buf(); for name in components { if matches!(name.to_str(), Some("resources" | "archive" | "inbox")) diff --git a/docs/vrs/requirements.md b/docs/vrs/requirements.md index 275c2c2e..984a956f 100644 --- a/docs/vrs/requirements.md +++ b/docs/vrs/requirements.md @@ -144,8 +144,30 @@ accepted. A durable incomplete-apply marker fences every declaration-plane snapshot and action after a crashed whole-catalog apply; a resident supervisor stays alive but performs zero lifecycle actions until the transaction is completed. + Every successful declaration commit advances a durable monotonic catalog + generation; whole-catalog apply advances it before its marker clears. Unlocked + diagnostic readers therefore detect even a completed declaration ABA across + their observation. A durable incomplete-generation intent fences readers + across each single-writer commit and is conservatively recovered by the next + exclusive writer. Writer staging exists only in the reserved control plane, + never among authoritative declaration leaves. Presence, messages, context, and Resource state remain independently writable and are never serialized behind catalog authoring. + A caller binds single-agent publication to the exact no-follow source capture + with an authoritative input digest. A canonical whole-catalog snapshot + externalizes the declaration-root digest while excluding runtime state and + workspace content. Its closed projection includes every regular file in a + bounded `_templates` library and exact declared canonical workspace directory + facts. Whole-catalog apply accepts only that projection, rechecks the root + digest under the exclusive lock, durably stages the desired bytes, and resumes + after interruption solely from a closed marker and its content-addressed + stage. Version 1 requires one explicit external PTY root and rejects effective + PTY-root changes. Fresh-catalog bootstrap is a separate cross-producer + transaction, not a catalog-apply mode. Apply never traverses, hashes, deletes, + or relocates workspace or runtime state. An absent canonical identity becomes + visible only as a complete bundle; a preexisting declared workspace skeleton + remains safe because the durable marker fences declaration readers and + marker-time state routing throughout leaf publication and verification. - **R23 Fail-closed task inventory:** One read-only machine command exposes every desired local PTY and exec task by agent identity, task name, runtime id, kind, lifecycle, retirement, desired state, runtime state, PID, creation @@ -160,6 +182,9 @@ accepted. incomplete; the external backend may already have recreated a concurrently removed registry. This diagnostic boundary is not transactionally serialized with catalog or runtime writers and is not control-plane cutover authority. + It samples the durable catalog generation and incomplete marker around + discovery and runtime observation; any marker, malformed fence, or generation + change makes the envelope incomplete. - **R24 Stable identity and bounded presentation:** The positional Agent Spec identity and its host-qualified bus identity remain the sole stable keys for routing, ownership, adoption, lifecycle, and automation. Agent Specs may diff --git a/docs/vrs/spec.md b/docs/vrs/spec.md index 939b60de..297840ac 100644 --- a/docs/vrs/spec.md +++ b/docs/vrs/spec.md @@ -95,8 +95,9 @@ st2 neither reads, writes, migrates, nor interprets it. 3. refuses declarations explicitly marked `meta { managed-by "nix" }`, unsupported formats, malformed catalogs, and ambiguous targets; 4. applies one span-bounded edit, reparses and validates the candidate; -5. fsyncs a same-directory temporary, rechecks the original inode/version and - bytes, atomically renames it, then fsyncs the declaration directory. +5. fsyncs a temporary under the reserved `.st2` control plane, rechecks the + original inode/version and bytes, atomically renames it through retained + no-follow directory capabilities, then fsyncs the declaration directory. `ST_AGENT` is a runner-provided convention in this trusted single-operator fleet, not an authenticated capability: a same-UID caller can alter or remove it, and @@ -155,9 +156,11 @@ and the portable Agent Spec envelope in ## Transactional catalog authoring +`st2 agent digest (--spec FILE | --bundle DIR)` captures a source through +retained no-follow file descriptors and returns its authoritative digest. `st2 agent publish --catalog ROOT (--spec FILE | --bundle DIR) -(--expect-absent | --expect-sha256 HEX) --json` is the sole supported -declaration writer. It accepts exactly one canonical KDL `agent` node with an +--input-sha256 HEX (--expect-absent | --expect-sha256 HEX) --json` binds +publication to that exact capture. It accepts exactly one canonical KDL `agent` node with an explicit, path-safe host and identity. st2 no longer exposes an intent compiler: external renderers own the transformation from human intent to exact Agent Spec bytes or a create-only publication bundle. @@ -168,18 +171,86 @@ read/write transaction domain: ```text publisher (EX) : snapshot input -> CAS -> full-catalog admission -> atomic publish + fsync reader (SH) : discover -> materialize/observe -> plan -> execute -state plane : message | context | Resource | status (unlocked) +bulk apply (EX) : root CAS -> durable stage+marker -> converge -> verify+clear +state plane : message | context | Resource | status (unlocked) ``` +Every publication temporary, including a not-yet-visible identity bundle, is +staged under `.st2`. Cross-directory rename therefore stays on the catalog +filesystem while a crash can leave debris only in the non-projected control +plane; declaration directories never contain writer-private leaves. + +Before a declaration writer mutates the live projection it durably creates +`.st2/catalog-generation-incomplete`. Shared declaration readers and read fences +fail closed while this intent exists. After the declaration and its parent are +durable, the writer advances and fsyncs `catalog-generation`, then clears and +fsyncs the intent. The next exclusive writer recovers an orphan intent by +conservatively advancing the generation before clearing it. A crash after the +advance but before intent removal may therefore skip a generation on recovery; +the contract is monotonic change detection, not an exactly-once counter. + +The lock file is a persistent real inode: replacing or removing it would split +the lock domain for a process that already has it open. Consequently, the first +coherent declaration reader may initialize exactly `.st2` and this lock even +when its requested operation later refuses. Refusal still performs no +declaration, workspace, or state mutation. + The publisher derives the destination from the captured declaration, replaces only `agent.kdl` for a hash-authorized update, and preserves all sibling runtime state. A bundle is create-only and is renamed from a hidden same-filesystem stage; retry reports `unchanged` only when every projected bundle file already matches. `--expect-absent` is idempotent for identical input. -`--expect-sha256` rejects a stale writer. Full-catalog admission rejects any +`--input-sha256` rejects a caller/source swap and `--expect-sha256` rejects a +stale declaration writer. Full-catalog admission rejects any structural validation error before publication. The typed result is `published` or `unchanged`. +`st2 catalog snapshot --catalog ROOT --output DIR --json` holds SH while it +captures the canonical declaration projection: `catalog.kdl`, exact +`agents///agent.kdl` files, static files inside those bounded +agent bundles, and every regular file in `_templates` whether or not a current +render references it. `_templates` is bounded to depth 8 below its root, 256 +files, 1 MiB per file, and 32 MiB total; symlinks, hard links, special nodes, +and reserved control/state names are rejected. Runtime state, `.git`, `.st2`, +the native `pty` registry, and workspace content are excluded. A +catalog-contained Agent `workspace` or Task `cwd` is valid only when it names +that agent bundle's canonical real `.workspace`; the empty directory itself is +an exact declaration fact, while its descendants are never traversed. The +classification uses launch-equivalent variable expansion, resolves relative +values from the Agent Spec bundle, and lexically normalizes before comparing +against the logical catalog. A relative spelling is accepted only when it +normalizes to that bundle's canonical `.workspace`; unresolved variables and +every other effective relative path fail closed. The +scanner always excludes a canonical `.workspace` subtree, including an orphan +left after an agent move or removal. External workspaces remain valid and are +not part of the projection. The output is a create-only durable directory; an +identical retry is `unchanged`. Its domain-separated, path-sorted root SHA-256 +covers normalized relative paths, file bytes, executable bits, and empty +workspace directory facts. + +`st2 catalog apply --catalog ROOT --prepared DIR --expect-sha256 HEX --json` +rejects any prepared state/control path, symlink, special node, unprojected +file/directory, malformed declaration, nonempty prepared workspace fact, +catalog-local/default PTY root, or effective PTY-root change. Hash-CAS captures +and validates exact prepared bytes, takes EX, rechecks the canonical live root, +and either reports `unchanged` for exact equality or creates a durable +content-addressed stage before publishing the marker. Version 1 requires an +explicit PTY root outside the canonical catalog. Fresh bootstrap is a separate +cross-producer transaction because catalog EX cannot reserve a PTY registry +against external producers. Hash-CAS permits declared live workspace facts and +their real ancestry to contain content. It changes +declaration leaves only; desired workspace facts must already exist, and +workspace content and canonical state are never traversed, deleted, or hashed. +When an identity path is absent, its complete bundle uses an exclusive +directory rename. When its declared workspace skeleton already exists, the +durable marker fences declaration readers and marker-time state routing until +every declaration leaf has been published and verified. Applied leaves and +their parents are fsynced, the live root is re-hashed and fully admitted, then +the marker is unlinked and `.st2` is fsynced. +The catalog parent is fsynced when `.st2` is first created, including the +concurrent create/observe race. Retained source capture rejects a staging +destination contained by its source before enumerating that source. + The lock file is never removed: replacing its inode would split the transaction domain for processes that already hold it open. Reconciliation holds SH from discovery through execution. Validation, doctor, roster, listing, @@ -192,7 +263,7 @@ transaction fence. Any presence is authoritative, including malformed content. The reserved canonical record is: ```json -{"schema":"st2.catalog-apply-incomplete.v1","desiredStorePath":"/nix/store/..."} +{"schema":"st2.catalog-apply-incomplete.v1","stageName":"catalog-apply-stage-","expectedRootSha256":"","preparedRootSha256":"","originalPaths":["", "..."]} ``` After taking its authoring lock, st2 refuses publication, validation, @@ -200,10 +271,26 @@ materialization, teardown, roster, doctor, and catalog listing while the marker exists. One-shot and selected reconcile fail explicitly. A resident supervisor instead remains alive, reports a skipped/incomplete pass, and performs no runtime observation or lifecycle action, avoiding a service restart storm. -Message, context, Resource, and status operations remain available. The future -whole-catalog apply command owns marker creation, admission, durable apply, -verification, and clearing inside one transaction; external lock execution and -bypass flags are not part of the contract. +Message, context, Resource, and status operations remain available. While the +marker exists they resolve canonical state from a validated address book: the +marker's original canonical agent keys union currently published real specs. +State-only directories are addressable only for original keys with recognized +real state; an incomplete or arbitrary new identity does not fall back to a +flat bus. Every host, identity, and message-box path is opened component by +component without following symlinks, and state mutations remain relative to +those retained capabilities. +A dotted bare identity is tried as +the complete local identity alongside every possible qualified bus-address +split; exactly one distinct canonical address must exist. Only real state +directories and a real regular status file can establish marker-time +addressability. Only `catalog apply --resume --catalog ROOT --json` may open an +existing marker. The closed marker and internal content-addressed stage are +sufficient recovery authority; the original prepared path and CAS precondition are +neither required nor consulted. Marker authority proves the original +precondition already passed, so recovery converges the partial live tree from +the durable desired stage and original owned-leaf list without re-enforcing +that stale precondition. Malformed or mismatched records remain fenced. +External lock execution and bypass flags are not part of the contract. ## Host-local scheduling and supervision @@ -300,9 +387,13 @@ validate ──► materialize ──► host-local st2 scheduler/reconciler time, and opaque generation id derived from stable backend evidence. Discovery runs before and after runtime observation. A semantic declaration - change across those passes makes the result incomplete. This detects - observed drift but does not serialize catalog writers or claim a - transactional snapshot. A runtime root positively absent at admission is + change across those passes makes the result incomplete. The reader also + samples `/.st2/catalog-generation` and the incomplete marker around + discovery and runtime observation. Every successful declaration writer + advances and fsyncs that monotonic generation after its durable commit; apply + does so after live verification and before clearing its marker. Even a + completed declaration ABA is therefore incomplete. This remains an observational + seqlock and does not serialize catalog writers. A runtime root positively absent at admission is empty and is not passed to its backend. An admitted PTY root that is removed or replaced during `pty list` is indeterminate; because the external backend creates an absent registry, concurrent root deletion is not a zero-write diff --git a/examples/README.md b/examples/README.md index 758913de..0aaed689 100644 --- a/examples/README.md +++ b/examples/README.md @@ -13,7 +13,9 @@ and gate the result before starting a process: ```sh st2 hooks install st2 hooks verify -st2 agent publish --catalog --bundle --expect-absent --json +input_sha256="$(st2 agent digest --bundle )" +st2 agent publish --catalog --bundle \ + --input-sha256 "$input_sha256" --expect-absent --json st2 validate --catalog st2 up --catalog --host --materialize-only st2 up --catalog --host --once diff --git a/examples/native/README.md b/examples/native/README.md index ae5dc942..9348375d 100644 --- a/examples/native/README.md +++ b/examples/native/README.md @@ -25,7 +25,9 @@ Use this sequence: ```sh st2 hooks install st2 hooks verify -st2 agent publish --catalog --bundle --expect-absent --json +input_sha256="$(st2 agent digest --bundle )" +st2 agent publish --catalog --bundle \ + --input-sha256 "$input_sha256" --expect-absent --json st2 validate st2 up --host --materialize-only st2 up --host --once diff --git a/src/agent_author.rs b/src/agent_author.rs index fd963dfd..6108223d 100644 --- a/src/agent_author.rs +++ b/src/agent_author.rs @@ -8,11 +8,10 @@ use std::collections::{BTreeMap, BTreeSet}; use std::fmt; -use std::fs::{self, OpenOptions}; +use std::fs; use std::io::Write as _; -use std::os::unix::fs::{MetadataExt as _, OpenOptionsExt as _, PermissionsExt as _}; +use std::os::unix::fs::{MetadataExt as _, PermissionsExt as _}; use std::path::{Path, PathBuf}; -use std::sync::atomic::{AtomicU64, Ordering}; use agent_spec::spec::{AGENT_DESCRIPTION_MAX_CHARS, AGENT_NAME_MAX_CHARS, validate_presentation}; use kdl::{KdlDocument, KdlNode}; @@ -20,8 +19,6 @@ use serde::Serialize; use crate::catalog_lock::CatalogLock; -static TMP_COUNTER: AtomicU64 = AtomicU64::new(0); - #[derive(Debug, Clone, Copy, PartialEq, Eq)] struct SourceVersion { device: u64, @@ -139,7 +136,7 @@ pub fn set_presentation( field: PresentationField, requested: Option<&str>, ) -> Result { - let _catalog_lock = CatalogLock::exclusive(catalog_root).map_err(|error| { + let catalog_lock = CatalogLock::exclusive(catalog_root).map_err(|error| { AuthorError::new( "catalog-lock-failed", format!("acquire catalog-authoring lock: {error:#}"), @@ -166,6 +163,10 @@ pub fn set_presentation( }) .transpose()?; let result = edit_declaration( + &catalog_lock, + catalog_root, + &crate::catalog_transaction::retained_dir_path(catalog_lock.control()) + .map_err(|error| AuthorError::new("declaration-write-failed", error.to_string()))?, &target.declaration, &target.identity, &target.source_host, @@ -286,7 +287,17 @@ fn edit_declaration_for_test( requested: Option<&str>, before_commit: impl FnOnce(), ) -> Result { + let control = path + .parent() + .expect("test declaration has a parent") + .join(crate::catalog_lock::CONTROL_DIR); + fs::create_dir_all(&control).expect("create test catalog control directory"); + let catalog_lock = CatalogLock::exclusive(path.parent().expect("test catalog has a parent")) + .expect("acquire test catalog lock"); edit_declaration( + &catalog_lock, + path.parent().expect("test catalog has a parent"), + &control, path, expected_identity, expected_host, @@ -298,6 +309,9 @@ fn edit_declaration_for_test( } fn edit_declaration( + catalog_lock: &CatalogLock, + catalog: &Path, + control: &Path, path: &Path, expected_identity: &str, expected_host: &str, @@ -368,6 +382,9 @@ fn edit_declaration( requested, )?; atomic_replace_checked( + catalog_lock, + catalog, + control, path, &original, original_version, @@ -710,6 +727,9 @@ fn verify_candidate( } fn atomic_replace_checked( + catalog_lock: &CatalogLock, + catalog: &Path, + control: &Path, path: &Path, original: &[u8], original_version: SourceVersion, @@ -723,28 +743,27 @@ fn atomic_replace_checked( format!("declaration path {} has no parent", path.display()), ) })?; - let temporary = directory.join(format!( - ".agent.kdl.presentation-{}-{}", - std::process::id(), - TMP_COUNTER.fetch_add(1, Ordering::Relaxed) - )); - let write = (|| -> std::io::Result<()> { - let mut file = OpenOptions::new() - .write(true) - .create_new(true) - .mode(mode) - .open(&temporary)?; - file.set_permissions(fs::Permissions::from_mode(mode))?; - file.write_all(replacement)?; - file.sync_all() - })(); - if let Err(error) = write { - let _ = fs::remove_file(&temporary); - return Err(AuthorError::new( - "declaration-write-failed", - format!("staging declaration {}: {error}", path.display()), - )); - } + let mut temporary = tempfile::Builder::new() + .prefix("agent-presentation-") + .tempfile_in(control) + .map_err(|error| { + AuthorError::new( + "declaration-write-failed", + format!("staging declaration {}: {error}", path.display()), + ) + })?; + temporary + .as_file_mut() + .set_permissions(fs::Permissions::from_mode(mode)) + .and_then(|()| temporary.write_all(replacement)) + .and_then(|()| temporary.as_file().sync_all()) + .map_err(|error| { + AuthorError::new( + "declaration-write-failed", + format!("staging declaration {}: {error}", path.display()), + ) + })?; + test_crash_after_temporary_write(); before_commit(); let current = fs::symlink_metadata(path) .ok() @@ -752,7 +771,6 @@ fn atomic_replace_checked( .map(|metadata| (SourceVersion::from_metadata(&metadata), fs::read(path).ok())); if !matches!(current, Some((version, Some(bytes))) if version == original_version && bytes == original) { - let _ = fs::remove_file(&temporary); return Err(AuthorError::new( "source-changed", format!( @@ -761,8 +779,18 @@ fn atomic_replace_checked( ), )); } - if let Err(error) = fs::rename(&temporary, path) { - let _ = fs::remove_file(&temporary); + let generation = catalog_lock.begin_generation_commit().map_err(|error| { + AuthorError::new( + "declaration-write-failed", + format!("prepare catalog generation: {error:#}"), + ) + })?; + if let Err(error) = crate::catalog_transaction::persist_tempfile_from_control( + catalog_lock.control(), + catalog, + temporary, + path, + ) { return Err(AuthorError::new( "declaration-write-failed", format!( @@ -771,7 +799,7 @@ fn atomic_replace_checked( ), )); } - fs::File::open(directory) + crate::catalog_transaction::open_dir_beneath(catalog, directory) .and_then(|directory| directory.sync_all()) .map_err(|error| { AuthorError::new( @@ -781,9 +809,26 @@ fn atomic_replace_checked( directory.display() ), ) - }) + })?; + generation.commit().map_err(|error| { + AuthorError::new( + "declaration-write-failed", + format!("advance catalog generation: {error:#}"), + ) + })?; + Ok(()) } +#[cfg(debug_assertions)] +fn test_crash_after_temporary_write() { + if std::env::var_os("ST2_TEST_AGENT_AUTHOR_CRASH_AFTER_TEMP").is_some() { + std::process::abort(); + } +} + +#[cfg(not(debug_assertions))] +fn test_crash_after_temporary_write() {} + #[cfg(test)] mod tests { use super::*; @@ -908,15 +953,8 @@ mod tests { ); let path = write(root, "h/worker/agent.kdl", &with_name); - let receipt = set_presentation( - root, - "h.worker", - "h", - None, - PresentationField::Name, - None, - ) - .unwrap(); + let receipt = + set_presentation(root, "h.worker", "h", None, PresentationField::Name, None).unwrap(); assert_eq!(receipt.result, AuthorOutcome::Changed); assert_eq!(fs::read_to_string(path).unwrap(), original); diff --git a/src/agent_publish.rs b/src/agent_publish.rs index ce9c495f..17c8b59c 100644 --- a/src/agent_publish.rs +++ b/src/agent_publish.rs @@ -1,12 +1,9 @@ //! Transactional publication of one canonical Agent Spec into a live catalog. -use std::collections::BTreeSet; -use std::ffi::CString; use std::fs::{self, File, OpenOptions}; -use std::io::Write; -use std::os::unix::ffi::OsStrExt; -use std::os::unix::fs::PermissionsExt; +use std::io::{Read as _, Write}; use std::os::unix::fs::symlink; +use std::os::unix::fs::{OpenOptionsExt as _, PermissionsExt}; use std::path::{Component, Path, PathBuf}; use agent_spec::discovery::parse_declared; @@ -16,8 +13,11 @@ use serde::Serialize; use sha2::{Digest, Sha256}; use crate::catalog_lock::CatalogLock; +use crate::catalog_transaction::sync_dir; const SCHEMA: &str = "st2.agent-publish.v1"; +const DIGEST_SCHEMA: &str = "st2.agent-source-digest.v1"; +const BUNDLE_DIGEST_DOMAIN: &[u8] = b"st2.agent-publish-bundle.v1\0"; #[derive(Debug, Clone)] pub enum PublishSource { @@ -36,6 +36,7 @@ pub struct PublishRequest { pub catalog: PathBuf, pub source: PublishSource, pub expectation: PublishExpectation, + pub input_sha256: String, } #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] @@ -52,6 +53,7 @@ pub struct PublishResult { pub status: PublishStatus, pub bus_id: String, pub path: PathBuf, + pub input_sha256: String, #[serde(skip_serializing_if = "Option::is_none")] pub before_sha256: Option, pub after_sha256: String, @@ -70,39 +72,48 @@ struct Candidate { bytes: Vec, host: String, identity: String, + input_sha256: String, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] +#[serde(rename_all = "lowercase")] +pub enum SourceKind { + Spec, + Bundle, +} + +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct SourceDigest { + pub schema: &'static str, + pub kind: SourceKind, + pub sha256: String, } impl Candidate { - fn stage(catalog: &Path, source: PublishSource) -> Result { - let control = catalog.join(crate::catalog_lock::CONTROL_DIR); + fn stage_in(parent: &Path, source: PublishSource) -> Result { let stage = tempfile::Builder::new() .prefix("agent-publish-") - .tempdir_in(&control) - .with_context(|| format!("stage publication in {}", control.display()))?; + .tempdir_in(parent) + .with_context(|| format!("stage publication in {}", parent.display()))?; let kind = match &source { PublishSource::Spec(path) => { - let metadata = fs::symlink_metadata(path) - .with_context(|| format!("read candidate spec {}", path.display()))?; + let mut input = open_regular_nofollow(path) + .with_context(|| format!("open candidate spec {}", path.display()))?; + let metadata = input.metadata()?; anyhow::ensure!( - metadata.is_file() && !metadata.file_type().is_symlink(), + metadata.is_file(), "candidate spec is not a regular file: {}", path.display() ); - let bytes = fs::read(path) - .with_context(|| format!("read candidate spec {}", path.display()))?; + let mut bytes = Vec::new(); + input.read_to_end(&mut bytes)?; write_synced(&stage.path().join("agent.kdl"), &bytes)?; CandidateKind::Spec } PublishSource::Bundle(path) => { - let metadata = fs::symlink_metadata(path) - .with_context(|| format!("read bundle {}", path.display()))?; - anyhow::ensure!( - metadata.is_dir() && !metadata.file_type().is_symlink(), - "bundle is not a real directory: {}", - path.display() - ); - validate_bundle_tree(path)?; - copy_tree(path, stage.path(), true)?; + crate::catalog_transaction::capture_real_tree(path, stage.path()) + .with_context(|| format!("capture bundle {}", path.display()))?; File::open(stage.path())?.sync_all()?; CandidateKind::Bundle } @@ -151,12 +162,17 @@ impl Candidate { .context("candidate must declare a non-empty explicit identity")?; validate_component("host", host)?; validate_component("identity", identity)?; + let input_sha256 = match kind { + CandidateKind::Spec => sha256(&bytes), + CandidateKind::Bundle => bundle_sha256(stage.path())?, + }; Ok(Self { stage, kind, bytes, host: host.to_string(), identity: identity.to_string(), + input_sha256, }) } @@ -165,14 +181,35 @@ impl Candidate { } } +pub fn digest_source(source: PublishSource) -> Result { + let parent = tempfile::tempdir().context("create source-digest staging root")?; + let candidate = Candidate::stage_in(parent.path(), source)?; + Ok(SourceDigest { + schema: DIGEST_SCHEMA, + kind: match candidate.kind { + CandidateKind::Spec => SourceKind::Spec, + CandidateKind::Bundle => SourceKind::Bundle, + }, + sha256: candidate.input_sha256, + }) +} + /// Publish one spec under the catalog's exclusive authoring lock. pub fn publish(request: PublishRequest) -> Result { + validate_sha256(&request.input_sha256)?; let catalog = request .catalog .canonicalize() .with_context(|| format!("canonicalize catalog {}", request.catalog.display()))?; - let _lock = CatalogLock::exclusive(&catalog)?; - let candidate = Candidate::stage(&catalog, request.source)?; + let lock = CatalogLock::exclusive(&catalog)?; + let control = crate::catalog_transaction::retained_dir_path(lock.control())?; + let candidate = Candidate::stage_in(&control, request.source)?; + anyhow::ensure!( + candidate.input_sha256 == request.input_sha256, + "publication input precondition failed: expected sha256 {}, captured {}", + request.input_sha256, + candidate.input_sha256 + ); let target_dir = catalog .join("agents") .join(&candidate.host) @@ -233,7 +270,7 @@ pub fn publish(request: PublishRequest) -> Result { } } - validate_overlay(&catalog, &candidate)?; + validate_overlay(&catalog, &control, &candidate)?; ensure_real_dir_chain( &catalog, target_dir @@ -258,30 +295,32 @@ pub fn publish(request: PublishRequest) -> Result { )); } + test_before_publication(); + let generation = lock.begin_generation_commit()?; + let published = result( + PublishStatus::Published, + &candidate, + target_spec.clone(), + before_hash, + after_hash, + ); match candidate.kind { CandidateKind::Spec => { ensure_real_dir_chain(&catalog, &target_dir)?; - atomic_write_spec(&target_spec, &candidate.bytes, before.is_some())?; + atomic_write_spec( + lock.control(), + &catalog, + &target_spec, + &candidate.bytes, + before.is_some(), + )?; } CandidateKind::Bundle => { - let result = result( - PublishStatus::Published, - &candidate, - target_spec, - before_hash, - after_hash, - ); - atomic_publish_staged_bundle(candidate.stage, &target_dir)?; - return Ok(result); + atomic_publish_staged_bundle(lock.control(), &catalog, candidate.stage, &target_dir)?; } } - Ok(result( - PublishStatus::Published, - &candidate, - target_spec, - before_hash, - after_hash, - )) + generation.commit()?; + Ok(published) } fn result( @@ -296,6 +335,7 @@ fn result( status, bus_id: candidate.bus_id(), path, + input_sha256: candidate.input_sha256.clone(), before_sha256, after_sha256, } @@ -347,8 +387,7 @@ fn read_regular_optional(path: &Path) -> Result>> { } } -fn validate_overlay(catalog: &Path, candidate: &Candidate) -> Result<()> { - let control = catalog.join(crate::catalog_lock::CONTROL_DIR); +fn validate_overlay(catalog: &Path, control: &Path, candidate: &Candidate) -> Result<()> { let shadow = tempfile::Builder::new() .prefix("catalog-admission-") .tempdir_in(&control) @@ -371,29 +410,8 @@ fn validate_overlay(catalog: &Path, candidate: &Candidate) -> Result<()> { CandidateKind::Bundle => overlay_tree(candidate.stage.path(), &target)?, } - let found = crate::discovery::discover(shadow.path()); - let hosts: BTreeSet<_> = found - .specs - .iter() - .filter_map(|spec| spec.host.clone()) - .collect(); - let mut errors = BTreeSet::new(); - for host in hosts { - let report = crate::validate::validate_for_host(shadow.path(), &host); - errors.extend( - report - .issues - .iter() - .filter(|issue| issue.severity == crate::validate::Severity::Error) - .map(|issue| format!("{} [{}]: {}", issue.path, issue.code, issue.message)), - ); - } - anyhow::ensure!( - errors.is_empty(), - "candidate fails full-catalog validation:\n{}", - errors.into_iter().collect::>().join("\n") - ); - Ok(()) + crate::catalog_transaction::validate_full_catalog(shadow.path()) + .context("candidate fails full-catalog validation") } fn copy_filtered_catalog( @@ -479,37 +497,44 @@ fn overlay_tree(source: &Path, destination: &Path) -> Result<()> { Ok(()) } -fn validate_bundle_tree(root: &Path) -> Result<()> { - anyhow::ensure!( - root.join("agent.kdl").is_file(), - "bundle must contain agent.kdl at its root" - ); - for entry in fs::read_dir(root).with_context(|| format!("read bundle {}", root.display()))? { - let path = entry?.path(); - let metadata = fs::symlink_metadata(&path)?; - if metadata.is_dir() { - validate_bundle_tree_entries(&path)?; - } else { - anyhow::ensure!( - metadata.is_file() && !metadata.file_type().is_symlink(), - "bundle contains a non-regular entry: {}", - path.display() - ); - } - } - Ok(()) +fn open_regular_nofollow(path: &Path) -> Result { + OpenOptions::new() + .read(true) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW | libc::O_NONBLOCK) + .open(path) + .with_context(|| format!("open regular file {}", path.display())) +} + +fn bundle_sha256(root: &Path) -> Result { + let mut hasher = Sha256::new(); + hasher.update(BUNDLE_DIGEST_DOMAIN); + hash_bundle_dir(root, root, &mut hasher)?; + Ok(format!("{:x}", hasher.finalize())) } -fn validate_bundle_tree_entries(root: &Path) -> Result<()> { - for entry in fs::read_dir(root).with_context(|| format!("read bundle {}", root.display()))? { - let path = entry?.path(); +fn hash_bundle_dir(root: &Path, dir: &Path, hasher: &mut Sha256) -> Result<()> { + for entry in sorted_entries(dir)? { + let path = entry.path(); + let relative = path + .strip_prefix(root)? + .to_str() + .context("bundle path is not UTF-8")? + .replace(std::path::MAIN_SEPARATOR, "/"); let metadata = fs::symlink_metadata(&path)?; - if metadata.is_dir() { - validate_bundle_tree_entries(&path)?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + hash_record(hasher, b'd', &relative, false, &[]); + hash_bundle_dir(root, &path, hasher)?; + } else if metadata.is_file() && !metadata.file_type().is_symlink() { + hash_record( + hasher, + b'f', + &relative, + metadata.permissions().mode() & 0o111 != 0, + &fs::read(&path)?, + ); } else { - anyhow::ensure!( - metadata.is_file() && !metadata.file_type().is_symlink(), - "bundle contains a non-regular entry: {}", + anyhow::bail!( + "staged bundle contains a symlink or special entry: {}", path.display() ); } @@ -517,28 +542,19 @@ fn validate_bundle_tree_entries(root: &Path) -> Result<()> { Ok(()) } -fn copy_tree(source: &Path, destination: &Path, sync: bool) -> Result<()> { - for entry in fs::read_dir(source).with_context(|| format!("read {}", source.display()))? { - let entry = entry?; - let from = entry.path(); - let to = destination.join(entry.file_name()); - let metadata = fs::symlink_metadata(&from)?; - if metadata.is_dir() { - fs::create_dir(&to)?; - copy_tree(&from, &to, sync)?; - if sync { - File::open(&to)?.sync_all()?; - } - } else if metadata.is_file() { - fs::copy(&from, &to)?; - if sync { - OpenOptions::new().read(true).open(&to)?.sync_all()?; - } - } else { - anyhow::bail!("unsupported bundle entry type: {}", from.display()); - } - } - Ok(()) +fn hash_record(hasher: &mut Sha256, kind: u8, path: &str, executable: bool, bytes: &[u8]) { + hasher.update([kind]); + hasher.update((path.len() as u64).to_be_bytes()); + hasher.update(path.as_bytes()); + hasher.update([u8::from(executable)]); + hasher.update((bytes.len() as u64).to_be_bytes()); + hasher.update(bytes); +} + +fn sorted_entries(path: &Path) -> Result> { + let mut entries = fs::read_dir(path)?.collect::>>()?; + entries.sort_by_key(fs::DirEntry::file_name); + Ok(entries) } fn ensure_real_dir_chain(catalog: &Path, target: &Path) -> Result<()> { @@ -572,31 +588,89 @@ fn ensure_real_dir_chain(catalog: &Path, target: &Path) -> Result<()> { Ok(()) } -fn atomic_write_spec(target: &Path, bytes: &[u8], replace: bool) -> Result<()> { +fn atomic_write_spec( + control_file: &File, + catalog: &Path, + target: &Path, + bytes: &[u8], + replace: bool, +) -> Result<()> { let parent = target.parent().context("spec target has no parent")?; + let control = crate::catalog_transaction::retained_dir_path(control_file)?; let mut temp = tempfile::Builder::new() - .prefix(".agent.kdl.publish-") - .tempfile_in(parent) - .with_context(|| format!("create temporary spec in {}", parent.display()))?; + .prefix("agent-publish-leaf-") + .tempfile_in(&control) + .with_context(|| format!("create temporary spec in {}", control.display()))?; temp.write_all(bytes)?; temp.as_file().sync_all()?; + test_crash_after_temporary_write(); if replace { - temp.persist(target) - .map_err(|error| error.error) - .with_context(|| format!("replace {}", target.display()))?; + crate::catalog_transaction::persist_tempfile_from_control( + control_file, + catalog, + temp, + target, + ) + .with_context(|| format!("replace {}", target.display()))?; } else { - fs::hard_link(temp.path(), target) - .with_context(|| format!("publish {}", target.display()))?; + crate::catalog_transaction::link_tempfile_from_control( + control_file, + catalog, + &temp, + target, + ) + .with_context(|| format!("publish {}", target.display()))?; temp.close()?; } - sync_dir(parent) + crate::catalog_transaction::open_dir_beneath(catalog, parent)? + .sync_all() + .map_err(Into::into) } -fn atomic_publish_staged_bundle(stage: tempfile::TempDir, target: &Path) -> Result<()> { +#[cfg(debug_assertions)] +fn test_crash_after_temporary_write() { + if std::env::var_os("ST2_TEST_AGENT_PUBLISH_CRASH_AFTER_TEMP").is_some() { + std::process::abort(); + } +} + +#[cfg(not(debug_assertions))] +fn test_crash_after_temporary_write() {} + +#[cfg(debug_assertions)] +fn test_before_publication() { + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_AGENT_PUBLISH_READY"), + std::env::var("ST2_TEST_AGENT_PUBLISH_RELEASE"), + ) else { + return; + }; + let _ = fs::write(ready, b"ready"); + while !Path::new(&release).exists() { + std::thread::yield_now(); + } +} + +#[cfg(not(debug_assertions))] +fn test_before_publication() {} + +fn atomic_publish_staged_bundle( + control: &File, + catalog: &Path, + stage: tempfile::TempDir, + target: &Path, +) -> Result<()> { let parent = target.parent().context("bundle target has no parent")?; - rename_noreplace(&stage.keep(), target) - .with_context(|| format!("publish bundle {}", target.display()))?; - sync_dir(parent) + crate::catalog_transaction::rename_noreplace_between_dirs( + control, + catalog, + stage.path(), + target, + ) + .with_context(|| format!("publish bundle {}", target.display()))?; + crate::catalog_transaction::open_dir_beneath(catalog, parent)? + .sync_all() + .map_err(Into::into) } fn bundle_projection_matches(source: &Path, target: &Path) -> Result { @@ -666,50 +740,6 @@ fn validate_existing_ancestry(catalog: &Path, target: &Path) -> Result<()> { Ok(()) } -fn rename_noreplace(source: &Path, target: &Path) -> std::io::Result<()> { - let source = CString::new(source.as_os_str().as_bytes()).map_err(|_| { - std::io::Error::new(std::io::ErrorKind::InvalidInput, "source contains NUL") - })?; - let target = CString::new(target.as_os_str().as_bytes()).map_err(|_| { - std::io::Error::new(std::io::ErrorKind::InvalidInput, "target contains NUL") - })?; - - #[cfg(any(target_os = "linux", target_os = "android"))] - let result = unsafe { - libc::renameat2( - libc::AT_FDCWD, - source.as_ptr(), - libc::AT_FDCWD, - target.as_ptr(), - libc::RENAME_NOREPLACE, - ) - }; - #[cfg(target_os = "macos")] - let result = unsafe { - libc::renameatx_np( - libc::AT_FDCWD, - source.as_ptr(), - libc::AT_FDCWD, - target.as_ptr(), - libc::RENAME_EXCL, - ) - }; - #[cfg(not(any(target_os = "linux", target_os = "android", target_os = "macos")))] - let result = { - let _ = (source, target); - return Err(std::io::Error::new( - std::io::ErrorKind::Unsupported, - "atomic no-replace directory rename is unsupported on this platform", - )); - }; - - if result == 0 { - Ok(()) - } else { - Err(std::io::Error::last_os_error()) - } -} - fn write_synced(path: &Path, bytes: &[u8]) -> Result<()> { let mut file = OpenOptions::new() .create_new(true) @@ -720,10 +750,3 @@ fn write_synced(path: &Path, bytes: &[u8]) -> Result<()> { file.sync_all()?; Ok(()) } - -fn sync_dir(path: &Path) -> Result<()> { - File::open(path) - .with_context(|| format!("open directory {}", path.display()))? - .sync_all() - .with_context(|| format!("sync directory {}", path.display())) -} diff --git a/src/catalog_lock.rs b/src/catalog_lock.rs index d2365ee6..c37105f2 100644 --- a/src/catalog_lock.rs +++ b/src/catalog_lock.rs @@ -14,6 +14,126 @@ use anyhow::{Context, Result}; pub const CONTROL_DIR: &str = ".st2"; pub const LOCK_FILE: &str = "catalog-authoring.lock"; pub const APPLY_MARKER: &str = "catalog-apply-incomplete"; +pub const GENERATION_FILE: &str = "catalog-generation"; +pub const GENERATION_INTENT_FILE: &str = "catalog-generation-incomplete"; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct CatalogReadFence(Option); + +pub fn read_fence(catalog: &Path) -> Result { + let first = read_generation(catalog)?; + ensure_authoring_complete(catalog)?; + let second = read_generation(catalog)?; + ensure_authoring_complete(catalog)?; + anyhow::ensure!( + first == second, + "catalog generation changed while sampling declaration state" + ); + Ok(CatalogReadFence(second)) +} + +fn advance_generation(control_file: &File) -> Result<()> { + let current = read_generation_from_control(control_file)?.unwrap_or(0); + let next = current + .checked_add(1) + .context("catalog generation counter exhausted")?; + let control = crate::catalog_transaction::retained_dir_path(control_file)?; + let target = control.join(GENERATION_FILE); + match fs::symlink_metadata(&target) { + Ok(metadata) => anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "catalog generation is not a real regular file: {}", + target.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error).context("inspect catalog generation"), + } + let mut temp = tempfile::Builder::new() + .prefix("catalog-generation-") + .tempfile_in(&control)?; + use std::io::Write as _; + writeln!(temp, "{next}")?; + temp.as_file().sync_all()?; + temp.persist(&target).map_err(|error| error.error)?; + control_file.sync_all()?; + Ok(()) +} + +fn read_generation(catalog: &Path) -> Result> { + let Some((control, _control_path)) = retained_control(catalog)? else { + return Ok(None); + }; + read_generation_from_control(&control) +} + +pub(crate) fn read_generation_token(catalog: &Path) -> Result> { + read_generation(catalog) +} + +fn read_generation_from_control(control: &File) -> Result> { + let control_path = crate::catalog_transaction::retained_dir_path(control)?; + let path = control_path.join(GENERATION_FILE); + let mut file = match OpenOptions::new() + .read(true) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW) + .open(&path) + { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error).context("open catalog generation"), + }; + let metadata = file.metadata()?; + anyhow::ensure!( + metadata.is_file(), + "catalog generation is not a real regular file: {}", + path.display() + ); + use std::io::Read as _; + let mut value = String::new(); + file.read_to_string(&mut value)?; + let value = value + .strip_suffix('\n') + .context("catalog generation is missing its newline terminator")?; + Ok(Some(value.parse().context("parse catalog generation")?)) +} + +fn ensure_authoring_complete(catalog: &Path) -> Result<()> { + let Some((_control, control_path)) = retained_control(catalog)? else { + return Ok(()); + }; + for (name, message) in [ + (APPLY_MARKER, "catalog apply is incomplete"), + ( + GENERATION_INTENT_FILE, + "catalog declaration commit is incomplete", + ), + ] { + let marker = control_path.join(name); + match fs::symlink_metadata(&marker) { + Ok(_) => anyhow::bail!("{message}: marker present at {}", marker.display()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => { + return Err(error) + .with_context(|| format!("inspect catalog marker {}", marker.display())); + } + } + } + Ok(()) +} + +fn retained_control(catalog: &Path) -> Result> { + let root = crate::catalog_transaction::open_dir_beneath(catalog, catalog)?; + let file = match crate::catalog_transaction::openat_dir_nofollow( + &root, + std::ffi::OsStr::new(CONTROL_DIR), + ) { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error).context("open catalog control directory"), + }; + let path = crate::catalog_transaction::retained_dir_path(&file)?; + Ok(Some((file, path))) +} #[derive(Debug, Clone, Copy)] enum Mode { @@ -25,18 +145,26 @@ enum Mode { #[derive(Debug)] pub struct CatalogLock { file: File, + control: File, } impl CatalogLock { pub fn shared(catalog: &Path) -> Result { - Self::acquire(catalog, Mode::Shared) + Self::acquire(catalog, Mode::Shared, false) } pub fn exclusive(catalog: &Path) -> Result { - Self::acquire(catalog, Mode::Exclusive) + Self::acquire(catalog, Mode::Exclusive, false) } - fn acquire(catalog: &Path, mode: Mode) -> Result { + /// The whole-catalog transaction is the only operation allowed to inspect and recover an + /// incomplete apply. Every other declaration reader/writer must keep using `shared` or + /// `exclusive`, which fail closed while the marker exists. + pub(crate) fn exclusive_for_catalog_apply(catalog: &Path) -> Result { + Self::acquire(catalog, Mode::Exclusive, true) + } + + fn acquire(catalog: &Path, mode: Mode, allow_incomplete_apply: bool) -> Result { let catalog = catalog .canonicalize() .with_context(|| format!("canonicalize catalog root {}", catalog.display()))?; @@ -49,15 +177,22 @@ impl CatalogLock { ); let control = catalog.join(CONTROL_DIR); - match fs::symlink_metadata(&control) { - Ok(metadata) => anyhow::ensure!( - metadata.is_dir() && !metadata.file_type().is_symlink(), - "catalog control path is not a real directory: {}", - control.display() - ), + let control_branch = match fs::symlink_metadata(&control) { + Ok(metadata) => { + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "catalog control path is not a real directory: {}", + control.display() + ); + "observed" + } Err(error) if error.kind() == std::io::ErrorKind::NotFound => { - match fs::create_dir(&control) { - Ok(()) => {} + test_control_creation_checkpoint(); + let branch = match fs::create_dir(&control) { + Ok(()) => { + test_control_created_checkpoint(); + "created" + } Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => { let metadata = fs::symlink_metadata(&control).with_context(|| { format!("re-read catalog control dir {}", control.display()) @@ -67,20 +202,38 @@ impl CatalogLock { "catalog control path is not a real directory: {}", control.display() ); + "raced" } Err(error) => { return Err(error).with_context(|| { format!("create catalog control dir {}", control.display()) }); } - } + }; + branch } Err(error) => { return Err(error) .with_context(|| format!("read catalog control dir {}", control.display())); } + }; + // The control directory is the durability root for the persistent lock, apply stage, and + // incomplete marker. Persist its catalog-parent entry before any caller can publish + // declaration leaves. This is unconditional after observing a real control dir: its creator + // may have crashed after mkdir but before its own parent fsync. + File::open(&catalog) + .with_context(|| format!("open catalog root {}", catalog.display()))? + .sync_all() + .with_context(|| format!("sync catalog root {}", catalog.display()))?; + #[cfg(debug_assertions)] + if let Ok(path) = std::env::var("ST2_TEST_CATALOG_CONTROL_BRANCH") { + let _ = fs::write(path, control_branch); } + let control_file = retained_control(&catalog)? + .context("catalog control directory disappeared while acquiring its lock")? + .0; + let control = crate::catalog_transaction::retained_dir_path(&control_file)?; let path = control.join(LOCK_FILE); let file = OpenOptions::new() .read(true) @@ -94,6 +247,12 @@ impl CatalogLock { Mode::Shared => libc::LOCK_SH, Mode::Exclusive => libc::LOCK_EX, }; + #[cfg(debug_assertions)] + if matches!(mode, Mode::Exclusive) + && let Ok(path) = std::env::var("ST2_TEST_CATALOG_LOCK_ATTEMPT") + { + let _ = fs::write(path, b"exclusive"); + } // SAFETY: `file` owns a valid descriptor for the duration of this call and the returned // guard. flock does not access Rust memory. let result = unsafe { libc::flock(file.as_raw_fd(), operation) }; @@ -101,22 +260,158 @@ impl CatalogLock { return Err(std::io::Error::last_os_error()) .with_context(|| format!("lock catalog authoring lock {}", path.display())); } - let marker = control.join(APPLY_MARKER); - match fs::symlink_metadata(&marker) { - Ok(_) => anyhow::bail!( - "catalog apply is incomplete: marker present at {}", - marker.display() - ), - Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} - Err(error) => { - return Err(error) - .with_context(|| format!("inspect catalog apply marker {}", marker.display())); + if !allow_incomplete_apply { + let marker = control.join(APPLY_MARKER); + match fs::symlink_metadata(&marker) { + Ok(_) => anyhow::bail!( + "catalog apply is incomplete: marker present at {}", + marker.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error).context("inspect catalog apply marker"), + } + } + if matches!(mode, Mode::Shared) { + let intent = control.join(GENERATION_INTENT_FILE); + match fs::symlink_metadata(&intent) { + Ok(_) => anyhow::bail!( + "catalog declaration commit is incomplete: marker present at {}", + intent.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error).context("inspect catalog generation intent"), } } - Ok(Self { file }) + let lock = Self { + file, + control: control_file, + }; + if matches!(mode, Mode::Exclusive) { + lock.recover_generation_intent()?; + test_lock_held_checkpoint(); + } + Ok(lock) + } + + pub(crate) fn advance_generation(&self) -> Result<()> { + advance_generation(&self.control) + } + + pub(crate) fn begin_generation_commit(&self) -> Result> { + let control = crate::catalog_transaction::retained_dir_path(&self.control)?; + let intent = control.join(GENERATION_INTENT_FILE); + let mut file = OpenOptions::new() + .write(true) + .create_new(true) + .mode(0o600) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW) + .open(&intent) + .context("create catalog generation intent")?; + use std::io::Write as _; + file.write_all(b"pending\n")?; + file.sync_all()?; + self.control.sync_all()?; + Ok(GenerationCommit { lock: self }) + } + + pub(crate) fn control(&self) -> &File { + &self.control + } + + fn recover_generation_intent(&self) -> Result<()> { + let control = crate::catalog_transaction::retained_dir_path(&self.control)?; + let intent = control.join(GENERATION_INTENT_FILE); + match fs::symlink_metadata(&intent) { + Ok(metadata) => anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "catalog generation intent is not a real regular file" + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()), + Err(error) => return Err(error).context("inspect catalog generation intent"), + } + self.advance_generation()?; + fs::remove_file(&intent).context("clear recovered catalog generation intent")?; + self.control.sync_all()?; + Ok(()) } } +#[cfg(debug_assertions)] +fn test_lock_held_checkpoint() { + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_CATALOG_LOCK_HELD_READY"), + std::env::var("ST2_TEST_CATALOG_LOCK_HELD_RELEASE"), + ) else { + return; + }; + let _ = fs::write(ready, b"ready"); + while !Path::new(&release).exists() { + std::thread::yield_now(); + } +} + +#[cfg(not(debug_assertions))] +fn test_lock_held_checkpoint() {} + +pub(crate) struct GenerationCommit<'a> { + lock: &'a CatalogLock, +} + +impl GenerationCommit<'_> { + pub(crate) fn commit(self) -> Result<()> { + #[cfg(debug_assertions)] + if std::env::var_os("ST2_TEST_GENERATION_FAIL_AFTER_COMMIT").is_some() { + anyhow::bail!("injected post-commit generation failure"); + } + self.lock.advance_generation()?; + let control = crate::catalog_transaction::retained_dir_path(&self.lock.control)?; + fs::remove_file(control.join(GENERATION_INTENT_FILE)) + .context("clear catalog generation intent")?; + self.lock.control.sync_all()?; + Ok(()) + } +} + +#[cfg(debug_assertions)] +fn test_control_creation_checkpoint() { + use std::time::{Duration, Instant}; + + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_CATALOG_CONTROL_READY"), + std::env::var("ST2_TEST_CATALOG_CONTROL_RELEASE"), + ) else { + return; + }; + let _ = fs::write(ready, b"ready"); + let deadline = Instant::now() + Duration::from_secs(5); + while !Path::new(&release).exists() && Instant::now() < deadline { + std::thread::sleep(Duration::from_millis(2)); + } +} + +#[cfg(not(debug_assertions))] +fn test_control_creation_checkpoint() {} + +#[cfg(debug_assertions)] +fn test_control_created_checkpoint() { + use std::time::{Duration, Instant}; + + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_CATALOG_CONTROL_CREATED_READY"), + std::env::var("ST2_TEST_CATALOG_CONTROL_CREATED_RELEASE"), + ) else { + return; + }; + let _ = fs::write(ready, b"ready"); + let deadline = Instant::now() + Duration::from_secs(5); + while !Path::new(&release).exists() && Instant::now() < deadline { + std::thread::sleep(Duration::from_millis(2)); + } +} + +#[cfg(not(debug_assertions))] +fn test_control_created_checkpoint() {} + impl Drop for CatalogLock { fn drop(&mut self) { // SAFETY: the descriptor remains valid until after Drop returns. diff --git a/src/catalog_transaction.rs b/src/catalog_transaction.rs new file mode 100644 index 00000000..9d2edb1f --- /dev/null +++ b/src/catalog_transaction.rs @@ -0,0 +1,2042 @@ +//! Canonical declaration snapshots and crash-recoverable whole-catalog application. +//! +//! This module owns the declaration projection shared by bulk readers and writers. Runtime state +//! is never copied, hashed, removed, or locked behind this transaction. + +use std::collections::{BTreeMap, BTreeSet}; +use std::fs::{self, File, OpenOptions}; +use std::io::{Read as _, Write as _}; +use std::os::fd::{AsRawFd as _, FromRawFd as _}; +use std::os::unix::fs::{MetadataExt as _, OpenOptionsExt as _, PermissionsExt as _}; +use std::path::{Component, Path, PathBuf}; + +use anyhow::{Context as _, Result}; +use serde::{Deserialize, Serialize}; +use sha2::{Digest as _, Sha256}; + +use crate::catalog_lock::{APPLY_MARKER, CONTROL_DIR, CatalogLock}; + +const SNAPSHOT_SCHEMA: &str = "st2.catalog-snapshot.v1"; +const APPLY_SCHEMA: &str = "st2.catalog-apply.v1"; +const MARKER_SCHEMA: &str = "st2.catalog-apply-incomplete.v1"; +const HASH_DOMAIN: &[u8] = b"st2.catalog-declaration-root.v1\0"; +const STAGE_PREFIX: &str = "catalog-apply-stage-"; +const WRITER_TEMP_PREFIXES: [&str; 3] = [ + ".agent.kdl.presentation-", + ".agent.kdl.publish-", + ".catalog-apply-file-", +]; +const TEMPLATE_MAX_DEPTH: usize = 8; +const TEMPLATE_MAX_FILES: usize = 256; +const TEMPLATE_MAX_FILE_BYTES: u64 = 1024 * 1024; +const TEMPLATE_MAX_TOTAL_BYTES: u64 = 32 * 1024 * 1024; + +#[derive(Debug)] +pub struct SnapshotRequest { + pub catalog: PathBuf, + pub output: PathBuf, +} + +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct SnapshotResult { + pub schema: &'static str, + pub status: SnapshotStatus, + pub catalog: PathBuf, + pub output: PathBuf, + pub root_sha256: String, + pub entries: usize, +} + +#[derive(Debug, Clone, Copy, Serialize)] +#[serde(rename_all = "lowercase")] +pub enum SnapshotStatus { + Created, + Unchanged, +} + +#[derive(Debug)] +pub struct ApplyRequest { + pub catalog: PathBuf, + pub mode: ApplyMode, +} + +#[derive(Debug)] +pub enum ApplyMode { + Prepared { + prepared: PathBuf, + expect_sha256: String, + }, + Resume, +} + +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct ApplyResult { + pub schema: &'static str, + pub status: ApplyStatus, + pub catalog: PathBuf, + pub prepared: Option, + pub before_sha256: String, + pub after_sha256: String, + pub entries: usize, + pub recovered: bool, +} + +#[derive(Debug, Clone, Copy, Serialize)] +#[serde(rename_all = "lowercase")] +pub enum ApplyStatus { + Applied, + Unchanged, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct ProjectedFile { + bytes: Vec, + executable: bool, +} + +#[derive(Debug, Clone)] +pub(crate) struct DeclarationProjection { + files: BTreeMap, + workspace_dirs: BTreeSet, + root_sha256: String, +} + +impl DeclarationProjection { + pub(crate) fn entries(&self) -> usize { + self.files.len() + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ProjectionSource { + Current, + Prepared, +} + +#[derive(Debug, Serialize, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ApplyMarker { + schema: String, + stage_name: String, + expected_root_sha256: String, + prepared_root_sha256: String, + original_paths: Vec, +} + +#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)] +pub struct AgentKey { + pub host: String, + pub identity: String, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CatalogTransition { + pub original_agents: BTreeSet, +} + +pub fn catalog_transition(catalog: &Path) -> Result> { + let root = open_dir_beneath(catalog, catalog)?; + let control = match openat_dir_nofollow(&root, std::ffi::OsStr::new(CONTROL_DIR)) { + Ok(control) => control, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error).context("open catalog control directory"), + }; + let marker = read_marker_optional(&retained_dir_path(&control)?.join(APPLY_MARKER))?; + let Some(marker) = marker else { + return Ok(None); + }; + validate_marker(&marker)?; + let original_agents = marker + .original_paths + .iter() + .filter_map(|path| { + let components = path.split('/').collect::>(); + (components.len() == 4 && components[0] == "agents" && components[3] == "agent.kdl") + .then(|| AgentKey { + host: components[1].to_string(), + identity: components[2].to_string(), + }) + }) + .collect(); + Ok(Some(CatalogTransition { original_agents })) +} + +/// Capture one coherent declaration plane under the shared catalog-authoring lock. +pub fn snapshot(request: SnapshotRequest) -> Result { + let catalog = canonical_real_dir(&request.catalog, "catalog")?; + let output = absolute_path(&request.output)?; + anyhow::ensure!( + !output.starts_with(&catalog), + "snapshot output must be outside the catalog: {}", + output.display() + ); + let parent = output.parent().context("snapshot output has no parent")?; + let parent = canonical_real_dir(parent, "snapshot output parent")?; + let output = parent.join( + output + .file_name() + .context("snapshot output has no final path component")?, + ); + + let _lock = CatalogLock::shared(&catalog)?; + let projection = project(&catalog, ProjectionSource::Current, &catalog)?; + validate_live_workspace_facts(&catalog, &projection.workspace_dirs)?; + match fs::symlink_metadata(&output) { + Ok(metadata) => { + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "snapshot output is not a real directory: {}", + output.display() + ); + let existing = project(&output, ProjectionSource::Prepared, &catalog)?; + anyhow::ensure!( + existing.root_sha256 == projection.root_sha256, + "snapshot output already exists with root sha256 {}, expected {}", + existing.root_sha256, + projection.root_sha256 + ); + return Ok(SnapshotResult { + schema: SNAPSHOT_SCHEMA, + status: SnapshotStatus::Unchanged, + catalog, + output, + entries: projection.entries(), + root_sha256: projection.root_sha256, + }); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => { + return Err(error) + .with_context(|| format!("inspect snapshot output {}", output.display())); + } + } + + let stage = tempfile::Builder::new() + .prefix(".st2-catalog-snapshot-") + .tempdir_in(&parent) + .with_context(|| format!("create snapshot stage in {}", parent.display()))?; + materialize_projection(&projection, stage.path())?; + rename_noreplace(&stage.keep(), &output) + .with_context(|| format!("publish snapshot {}", output.display()))?; + sync_dir(&parent)?; + + Ok(SnapshotResult { + schema: SNAPSHOT_SCHEMA, + status: SnapshotStatus::Created, + catalog, + output, + entries: projection.entries(), + root_sha256: projection.root_sha256, + }) +} + +/// Apply a complete prepared declaration plane under one exclusive transaction. +pub fn apply(request: ApplyRequest) -> Result { + let catalog = canonical_real_dir(&request.catalog, "catalog")?; + let prepared_input = match request.mode { + ApplyMode::Prepared { + prepared, + expect_sha256, + } => { + validate_sha256(&expect_sha256)?; + let prepared = canonical_real_dir_no_alias(&prepared, "prepared catalog")?; + anyhow::ensure!( + !prepared.starts_with(&catalog), + "prepared catalog must be outside the live catalog: {}", + prepared.display() + ); + let captured = tempfile::tempdir().context("create prepared-catalog capture root")?; + capture_prepared_catalog(&prepared, captured.path())?; + let desired = project(captured.path(), ProjectionSource::Prepared, &catalog)?; + Some((prepared, expect_sha256, desired)) + } + ApplyMode::Resume => None, + }; + + let lock = CatalogLock::exclusive_for_catalog_apply(&catalog)?; + let control = retained_dir_path(lock.control())?; + cleanup_writer_temporaries(&catalog)?; + let marker_path = control.join(APPLY_MARKER); + let existing_marker = read_marker_optional(&marker_path)?; + let recovered = existing_marker.is_some(); + let (prepared, expect_sha256, desired, marker) = match (prepared_input, existing_marker) { + (Some(_), Some(_)) => { + anyhow::bail!("catalog apply is incomplete; recover only with `catalog apply --resume`") + } + (Some((prepared, expect_sha256, desired)), None) => { + (Some(prepared), expect_sha256, desired, None) + } + (None, Some(marker)) => { + validate_marker(&marker)?; + let stage_path = control.join(&marker.stage_name); + let staged = project(&stage_path, ProjectionSource::Prepared, &catalog) + .context("validate durable recovery stage")?; + anyhow::ensure!( + staged.root_sha256 == marker.prepared_root_sha256, + "durable recovery stage hash mismatch: expected {}, found {}", + marker.prepared_root_sha256, + staged.root_sha256 + ); + ( + None, + marker.expected_root_sha256.clone(), + staged, + Some(marker), + ) + } + (None, None) => anyhow::bail!("catalog apply --resume requires an incomplete apply marker"), + }; + + validate_live_workspace_facts(&catalog, &desired.workspace_dirs)?; + // Admission reads exact durable/captured declaration bytes. Catalog-contained workspace facts + // are mirrored as empty directories; their live content is never copied or hashed. + let admission = tempfile::tempdir().context("create prepared-catalog admission root")?; + materialize_projection(&desired, admission.path())?; + validate_full_catalog(admission.path())?; + let desired_config = crate::catalog::load(admission.path())?; + validate_external_pty_root(&catalog, &desired_config)?; + + let stage_name = stage_name(&desired.root_sha256); + let stage_path = control.join(&stage_name); + let (before_sha256, original_paths, current) = if let Some(marker) = marker { + anyhow::ensure!( + marker.stage_name == stage_name, + "incomplete catalog apply stage name does not match its prepared root" + ); + (expect_sha256.clone(), marker.original_paths, None) + } else { + let current = project_excluding( + &catalog, + ProjectionSource::Current, + &catalog, + &desired.workspace_dirs, + )?; + let live_config = crate::catalog::load(&catalog)?; + let same_pty_root = effective_pty_root(&catalog, &live_config) + == effective_pty_root(&catalog, &desired_config); + if current.root_sha256 == desired.root_sha256 && same_pty_root { + return Ok(ApplyResult { + schema: APPLY_SCHEMA, + status: ApplyStatus::Unchanged, + catalog, + prepared, + before_sha256: current.root_sha256.clone(), + entries: desired.entries(), + after_sha256: desired.root_sha256, + recovered: false, + }); + } + anyhow::ensure!( + same_pty_root, + "catalog apply v1 refuses an effective pty-root change" + ); + anyhow::ensure!( + current.root_sha256 == expect_sha256, + "catalog apply precondition failed: expected sha256 {}, found {}", + expect_sha256, + current.root_sha256 + ); + let original_paths = current.files.keys().cloned().collect::>(); + ensure_durable_stage(lock.control(), &catalog, &stage_name, &desired)?; + write_marker( + lock.control(), + &ApplyMarker { + schema: MARKER_SCHEMA.to_string(), + stage_name: stage_name.clone(), + expected_root_sha256: expect_sha256.clone(), + prepared_root_sha256: desired.root_sha256.clone(), + original_paths: original_paths.clone(), + }, + )?; + test_checkpoint("marker-created"); + (current.root_sha256.clone(), original_paths, Some(current)) + }; + + let staged = project(&stage_path, ProjectionSource::Prepared, &catalog)?; + let generation = lock.begin_generation_commit()?; + apply_projection( + lock.control(), + &catalog, + &original_paths, + current.as_ref(), + &staged, + )?; + test_checkpoint("before-verify"); + let verified = project(&catalog, ProjectionSource::Current, &catalog)?; + anyhow::ensure!( + verified.root_sha256 == staged.root_sha256, + "catalog apply verification failed: expected {}, found {}", + staged.root_sha256, + verified.root_sha256 + ); + validate_full_catalog(&catalog).context("validate applied live catalog")?; + sync_dir(&catalog)?; + generation.commit()?; + test_checkpoint("before-clear"); + fs::remove_file(&marker_path) + .with_context(|| format!("clear catalog apply marker {}", marker_path.display()))?; + lock.control().sync_all()?; + let _ = fs::remove_dir_all(&stage_path); + let _ = lock.control().sync_all(); + + Ok(ApplyResult { + schema: APPLY_SCHEMA, + status: ApplyStatus::Applied, + catalog, + prepared, + before_sha256, + entries: verified.entries(), + after_sha256: verified.root_sha256, + recovered, + }) +} + +/// Full structural and host-scoped validation for a complete prospective catalog. +pub(crate) fn validate_full_catalog(root: &Path) -> Result<()> { + let found = crate::discover(root); + let mut hosts = BTreeSet::new(); + for spec in &found.specs { + let host = spec + .host + .as_deref() + .context("canonical declaration is missing explicit host")?; + hosts.insert(host.to_string()); + } + let mut errors = BTreeSet::new(); + let report = crate::validate::validate(root); + errors.extend( + report + .issues + .iter() + .filter(|issue| issue.severity == crate::validate::Severity::Error) + .map(format_issue), + ); + for host in hosts { + let report = crate::validate::validate_for_host(root, &host); + errors.extend( + report + .issues + .iter() + .filter(|issue| issue.severity == crate::validate::Severity::Error) + .map(format_issue), + ); + } + anyhow::ensure!( + errors.is_empty(), + "catalog fails full validation:\n{}", + errors.into_iter().collect::>().join("\n") + ); + Ok(()) +} + +fn format_issue(issue: &crate::validate::Issue) -> String { + format!("{} [{}]: {}", issue.path, issue.code, issue.message) +} + +fn project( + root: &Path, + source: ProjectionSource, + logical_catalog: &Path, +) -> Result { + project_excluding(root, source, logical_catalog, &BTreeSet::new()) +} + +fn project_excluding( + root: &Path, + source: ProjectionSource, + logical_catalog: &Path, + additional_workspace_dirs: &BTreeSet, +) -> Result { + let metadata = fs::symlink_metadata(root)?; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "projection root is not a real directory: {}", + root.display() + ); + let mut files = BTreeMap::new(); + add_optional_regular(root, &root.join(crate::catalog::CONFIG_FILE), &mut files)?; + let spec_paths = collect_canonical_specs(root, source, &mut files)?; + let discovered = crate::discover(root); + let mut specs = Vec::new(); + for path in &spec_paths { + let declared = crate::discovery::parse_declared(path) + .with_context(|| format!("parse canonical declaration {}", path.display()))?; + anyhow::ensure!( + declared.len() == 1, + "canonical declaration must contain exactly one agent: {}", + path.display() + ); + let relative = path.strip_prefix(root)?; + let components = normal_components(relative)?; + let expected_host = components[1].as_str(); + let expected_identity = components[2].as_str(); + anyhow::ensure!( + declared[0].host.as_deref() == Some(expected_host) + && declared[0].identity.as_deref() == Some(expected_identity), + "canonical declaration needs explicit host '{}' and identity '{}': {}", + expected_host, + expected_identity, + path.display() + ); + let matching = discovered + .specs + .iter() + .filter(|spec| spec.path == *path) + .collect::>(); + anyhow::ensure!( + matching.len() == 1, + "canonical declaration did not lower to exactly one agent: {}", + path.display() + ); + specs.push(matching[0].clone()); + } + + let workspace_dirs = catalog_workspace_dirs(root, logical_catalog, &specs)?; + let mut scan_exclusions = workspace_dirs.clone(); + scan_exclusions.extend(additional_workspace_dirs.iter().cloned()); + for spec in &specs { + let bundle = spec.path.parent().context("canonical spec has no bundle")?; + collect_bundle_files(root, bundle, bundle, source, &scan_exclusions, &mut files)?; + } + collect_templates(root, source, &mut files)?; + + for spec in &specs { + let host = spec.host.as_deref().context("explicit host disappeared")?; + for input in crate::materialize::catalog_owned_render_inputs(root, spec, host)? { + let relative = normalized_relative(root, &input)?; + let in_bundle = spec + .path + .parent() + .is_some_and(|parent| input.starts_with(parent)); + anyhow::ensure!( + in_bundle || relative.starts_with("_templates/"), + "catalog-owned render input must be inside its agent bundle or _templates: {}", + input.display() + ); + add_regular(root, &input, &mut files)?; + } + } + + if source == ProjectionSource::Prepared { + validate_prepared_workspace_facts(root, &workspace_dirs)?; + reject_unprojected_entries(root, &files, &workspace_dirs)?; + } + let root_sha256 = hash_projection(&files, &workspace_dirs); + Ok(DeclarationProjection { + files, + workspace_dirs, + root_sha256, + }) +} + +fn collect_canonical_specs( + root: &Path, + source: ProjectionSource, + files: &mut BTreeMap, +) -> Result> { + let agents = root.join("agents"); + let Some(host_entries) = read_real_dir_optional(&agents)? else { + return Ok(Vec::new()); + }; + let mut specs = Vec::new(); + for host_entry in host_entries { + let host_path = host_entry.path(); + ensure_safe_component(&host_entry.file_name(), "host")?; + ensure_real_dir(&host_path, "canonical host directory")?; + for identity_entry in sorted_entries(&host_path)? { + let identity_path = identity_entry.path(); + ensure_safe_component(&identity_entry.file_name(), "identity")?; + ensure_real_dir(&identity_path, "canonical identity directory")?; + let spec = identity_path.join("agent.kdl"); + match fs::symlink_metadata(&spec) { + Ok(metadata) => { + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "canonical spec is not a real regular file: {}", + spec.display() + ); + add_regular(root, &spec, files)?; + specs.push(spec); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + if source == ProjectionSource::Prepared { + reject_state_children(&identity_path)?; + } + } + Err(error) => { + return Err(error) + .with_context(|| format!("inspect canonical spec {}", spec.display())); + } + } + } + } + specs.sort(); + Ok(specs) +} + +fn collect_bundle_files( + root: &Path, + bundle_root: &Path, + dir: &Path, + source: ProjectionSource, + workspace_dirs: &BTreeSet, + files: &mut BTreeMap, +) -> Result<()> { + for entry in sorted_entries(dir)? { + let path = entry.path(); + let relative = normalized_relative(root, &path)?; + let relative_to_bundle = path.strip_prefix(bundle_root)?; + let first = relative_to_bundle.components().next(); + let name = entry.file_name(); + let name_text = name.to_str().context("bundle path is not UTF-8")?; + if is_writer_temporary(name_text) { + let metadata = fs::symlink_metadata(&path)?; + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "catalog writer temporary is not a real regular file: {}", + path.display() + ); + anyhow::ensure!( + source == ProjectionSource::Current, + "prepared catalog contains a reserved writer temporary: {}", + path.display() + ); + continue; + } + let canonical_workspace = first.is_some() + && relative_to_bundle.components().count() == 1 + && name_text == ".workspace"; + if canonical_workspace || workspace_dirs.contains(&relative) { + continue; + } + let state = matches!(name_text, "resources" | "archive" | "inbox" | "status") + || name_text.starts_with(".status.tmp-"); + if first.is_some() && relative_to_bundle.components().count() == 1 && state { + if source == ProjectionSource::Prepared { + anyhow::bail!( + "prepared catalog contains state-plane path: {}", + path.display() + ); + } + continue; + } + anyhow::ensure!( + !matches!(name_text, ".git" | ".st2"), + "bundle contains reserved control path: {}", + path.display() + ); + let metadata = fs::symlink_metadata(&path)?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + collect_bundle_files(root, bundle_root, &path, source, workspace_dirs, files)?; + } else if metadata.is_file() && !metadata.file_type().is_symlink() { + add_regular(root, &path, files)?; + } else { + anyhow::bail!( + "catalog declaration plane contains symlink or special entry: {}", + path.display() + ); + } + } + Ok(()) +} + +fn catalog_workspace_dirs( + root: &Path, + logical_catalog: &Path, + specs: &[agent_spec::spec::AgentSpec], +) -> Result> { + let mut facts = BTreeSet::new(); + for spec in specs { + let bundle = spec + .path + .parent() + .context("canonical spec has no bundle directory")?; + let bundle_relative = normalized_relative(root, bundle)?; + let logical_bundle = logical_catalog.join(&bundle_relative); + let expected = format!("{bundle_relative}/.workspace"); + let paths = spec + .workspace + .iter() + .map(String::as_str) + .chain(spec.tasks.iter().filter_map(|task| task.cwd.as_deref())); + for raw in paths { + let resolved = crate::expand::resolve_spec_path(raw, logical_catalog, &logical_bundle)?; + if resolved.starts_with(logical_catalog) { + let relative = normalized_relative(logical_catalog, &resolved)?; + anyhow::ensure!( + relative == expected, + "catalog-contained workspace/cwd must use canonical {}: {}", + expected, + resolved.display() + ); + facts.insert(relative); + } + } + } + Ok(facts) +} + +fn validate_prepared_workspace_facts(root: &Path, facts: &BTreeSet) -> Result<()> { + for relative in facts { + let path = root.join(relative); + ensure_real_dir_chain_present(root, &path, "prepared workspace fact")?; + anyhow::ensure!( + sorted_entries(&path)?.is_empty(), + "prepared workspace fact must be an empty directory: {}", + path.display() + ); + } + Ok(()) +} + +fn collect_templates( + root: &Path, + source: ProjectionSource, + files: &mut BTreeMap, +) -> Result<()> { + let templates = root.join("_templates"); + let Some(_) = read_real_dir_optional(&templates)? else { + return Ok(()); + }; + let mut count = 0usize; + let mut total = 0u64; + collect_template_dir( + root, &templates, &templates, source, 0, &mut count, &mut total, files, + ) +} + +fn collect_template_dir( + root: &Path, + templates: &Path, + dir: &Path, + source: ProjectionSource, + depth: usize, + count: &mut usize, + total: &mut u64, + files: &mut BTreeMap, +) -> Result<()> { + anyhow::ensure!( + depth <= TEMPLATE_MAX_DEPTH, + "_templates exceeds maximum depth {TEMPLATE_MAX_DEPTH}" + ); + for entry in sorted_entries(dir)? { + let path = entry.path(); + let name = entry.file_name(); + let name = name.to_str().context("_templates path is not UTF-8")?; + if is_writer_temporary(name) { + let metadata = fs::symlink_metadata(&path)?; + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "catalog writer temporary is not a real regular file: {}", + path.display() + ); + anyhow::ensure!( + source == ProjectionSource::Current, + "prepared catalog contains a reserved writer temporary: {}", + path.display() + ); + continue; + } + anyhow::ensure!( + !matches!( + name, + ".git" | ".st2" | "pty" | "resources" | "archive" | "inbox" | "status" + ), + "_templates contains reserved control/state path: {}", + path.display() + ); + let metadata = fs::symlink_metadata(&path)?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + collect_template_dir( + root, + templates, + &path, + source, + depth + 1, + count, + total, + files, + )?; + } else if metadata.is_file() && !metadata.file_type().is_symlink() { + let relative_depth = path.strip_prefix(templates)?.components().count(); + anyhow::ensure!( + relative_depth <= TEMPLATE_MAX_DEPTH, + "_templates file exceeds maximum depth {TEMPLATE_MAX_DEPTH}: {}", + path.display() + ); + anyhow::ensure!( + metadata.nlink() == 1, + "_templates file must not be hard-linked: {}", + path.display() + ); + anyhow::ensure!( + metadata.len() <= TEMPLATE_MAX_FILE_BYTES, + "_templates file exceeds {TEMPLATE_MAX_FILE_BYTES} bytes: {}", + path.display() + ); + *count += 1; + *total = total.saturating_add(metadata.len()); + anyhow::ensure!( + *count <= TEMPLATE_MAX_FILES, + "_templates exceeds {TEMPLATE_MAX_FILES} files" + ); + anyhow::ensure!( + *total <= TEMPLATE_MAX_TOTAL_BYTES, + "_templates exceeds {TEMPLATE_MAX_TOTAL_BYTES} total bytes" + ); + add_regular(root, &path, files)?; + } else { + anyhow::bail!( + "_templates contains a symlink or special entry: {}", + path.display() + ); + } + } + Ok(()) +} + +fn reject_state_children(identity_path: &Path) -> Result<()> { + for entry in sorted_entries(identity_path)? { + let name = entry.file_name(); + let name = name.to_str().context("identity path is not UTF-8")?; + if matches!(name, "resources" | "archive" | "inbox" | "status") + || name.starts_with(".status.tmp-") + { + anyhow::bail!( + "prepared catalog contains state-plane path: {}", + entry.path().display() + ); + } + } + Ok(()) +} + +fn add_optional_regular( + root: &Path, + path: &Path, + files: &mut BTreeMap, +) -> Result<()> { + match fs::symlink_metadata(path) { + Ok(_) => add_regular(root, path, files), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(error) => Err(error).with_context(|| format!("inspect {}", path.display())), + } +} + +fn add_regular( + root: &Path, + path: &Path, + files: &mut BTreeMap, +) -> Result<()> { + let metadata = fs::symlink_metadata(path)?; + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "declaration input is not a real regular file: {}", + path.display() + ); + let relative = normalized_relative(root, path)?; + let bytes = + fs::read(path).with_context(|| format!("read declaration input {}", path.display()))?; + let executable = metadata.permissions().mode() & 0o111 != 0; + files.insert(relative, ProjectedFile { bytes, executable }); + Ok(()) +} + +fn reject_unprojected_entries( + root: &Path, + files: &BTreeMap, + workspace_dirs: &BTreeSet, +) -> Result<()> { + let mut allowed_dirs = BTreeSet::new(); + for path in files.keys() { + let mut parent = Path::new(path).parent(); + while let Some(value) = parent { + if value.as_os_str().is_empty() { + break; + } + allowed_dirs.insert(value.to_string_lossy().to_string()); + parent = value.parent(); + } + } + for path in workspace_dirs { + let mut value = Some(Path::new(path)); + while let Some(dir) = value { + if dir.as_os_str().is_empty() { + break; + } + allowed_dirs.insert(dir.to_string_lossy().to_string()); + value = dir.parent(); + } + } + reject_unprojected_recursive(root, root, files, &allowed_dirs) +} + +fn reject_unprojected_recursive( + root: &Path, + dir: &Path, + files: &BTreeMap, + allowed_dirs: &BTreeSet, +) -> Result<()> { + for entry in sorted_entries(dir)? { + let path = entry.path(); + let relative = normalized_relative(root, &path)?; + let metadata = fs::symlink_metadata(&path)?; + anyhow::ensure!( + !metadata.file_type().is_symlink(), + "prepared catalog contains a symlink: {}", + path.display() + ); + if metadata.is_dir() { + anyhow::ensure!( + allowed_dirs.contains(&relative), + "prepared catalog contains unprojected directory: {}", + path.display() + ); + reject_unprojected_recursive(root, &path, files, allowed_dirs)?; + } else if metadata.is_file() { + anyhow::ensure!( + files.contains_key(&relative), + "prepared catalog contains unprojected file: {}", + path.display() + ); + } else { + anyhow::bail!( + "prepared catalog contains a special entry: {}", + path.display() + ); + } + } + Ok(()) +} + +fn hash_projection( + files: &BTreeMap, + workspace_dirs: &BTreeSet, +) -> String { + let mut hasher = Sha256::new(); + hasher.update(HASH_DOMAIN); + for (path, file) in files { + hasher.update([1]); + hasher.update((path.len() as u64).to_be_bytes()); + hasher.update(path.as_bytes()); + hasher.update([u8::from(file.executable)]); + hasher.update((file.bytes.len() as u64).to_be_bytes()); + hasher.update(&file.bytes); + } + for path in workspace_dirs { + hasher.update([2]); + hasher.update((path.len() as u64).to_be_bytes()); + hasher.update(path.as_bytes()); + } + format!("{:x}", hasher.finalize()) +} + +fn materialize_projection(projection: &DeclarationProjection, root: &Path) -> Result<()> { + for (relative, file) in &projection.files { + let target = root.join(relative); + let parent = target.parent().context("projected file has no parent")?; + ensure_real_dir_chain(root, parent)?; + let mut output = OpenOptions::new() + .create_new(true) + .write(true) + .mode(if file.executable { 0o755 } else { 0o644 }) + .open(&target) + .with_context(|| format!("create projected file {}", target.display()))?; + output.write_all(&file.bytes)?; + output.sync_all()?; + } + for relative in &projection.workspace_dirs { + ensure_real_dir_chain(root, &root.join(relative))?; + } + sync_tree_dirs(root) +} + +fn ensure_durable_stage( + control_file: &File, + catalog: &Path, + stage_name: &str, + desired: &DeclarationProjection, +) -> Result<()> { + let control = retained_dir_path(control_file)?; + let stage_path = control.join(stage_name); + match fs::symlink_metadata(&stage_path) { + Ok(metadata) => { + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "catalog apply stage is not a real directory: {}", + stage_path.display() + ); + let existing = project(&stage_path, ProjectionSource::Prepared, catalog)?; + anyhow::ensure!( + existing.root_sha256 == desired.root_sha256 + && existing.workspace_dirs == desired.workspace_dirs, + "existing catalog apply stage has unexpected root sha256" + ); + return Ok(()); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + let stage = tempfile::Builder::new() + .prefix(".catalog-apply-stage-") + .tempdir_in(&control)?; + materialize_projection(desired, stage.path())?; + let source = stage.keep(); + renameat_noreplace( + control_file, + source.file_name().context("stage has no name")?, + control_file, + std::ffi::OsStr::new(stage_name), + )?; + control_file.sync_all().map_err(Into::into) +} + +fn is_writer_temporary(name: &str) -> bool { + WRITER_TEMP_PREFIXES + .iter() + .any(|prefix| name.starts_with(prefix)) +} + +fn cleanup_writer_temporaries(catalog: &Path) -> Result<()> { + cleanup_writer_temporaries_in(catalog, false, false)?; + let agents = catalog.join("agents"); + if let Some(hosts) = read_real_dir_optional(&agents)? { + for host in hosts { + ensure_safe_component(&host.file_name(), "host")?; + ensure_real_dir(&host.path(), "canonical host directory")?; + for identity in sorted_entries(&host.path())? { + ensure_safe_component(&identity.file_name(), "identity")?; + ensure_real_dir(&identity.path(), "canonical identity directory")?; + cleanup_writer_temporaries_in(&identity.path(), true, true)?; + } + } + } + let templates = catalog.join("_templates"); + if read_real_dir_optional(&templates)?.is_some() { + cleanup_writer_temporaries_in(&templates, true, false)?; + } + Ok(()) +} + +fn cleanup_writer_temporaries_in(dir: &Path, recursive: bool, identity_root: bool) -> Result<()> { + for entry in sorted_entries(dir)? { + let path = entry.path(); + let name = entry + .file_name() + .into_string() + .map_err(|_| anyhow::anyhow!("catalog path is not UTF-8: {}", path.display()))?; + let metadata = fs::symlink_metadata(&path)?; + if is_writer_temporary(&name) { + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "catalog writer temporary is not a real regular file: {}", + path.display() + ); + fs::remove_file(&path).with_context(|| { + format!("remove stale catalog writer temporary {}", path.display()) + })?; + sync_dir(dir)?; + continue; + } + if !recursive || !metadata.is_dir() || metadata.file_type().is_symlink() { + continue; + } + if identity_root + && matches!( + name.as_str(), + ".workspace" | "resources" | "archive" | "inbox" + ) + { + continue; + } + cleanup_writer_temporaries_in(&path, true, false)?; + } + Ok(()) +} + +fn apply_projection( + control: &File, + catalog: &Path, + original_paths: &[String], + current: Option<&DeclarationProjection>, + desired: &DeclarationProjection, +) -> Result<()> { + let atomically_created = create_new_identity_bundles(control, catalog, desired)?; + let mut stale = original_paths + .iter() + .filter(|path| !desired.files.contains_key(*path)) + .cloned() + .collect::>(); + stale.sort_by_key(|path| (is_canonical_agent_spec(path), path.clone())); + for path in stale { + let target = catalog.join(&path); + let metadata = match fs::symlink_metadata(&target) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => continue, + Err(error) => return Err(error.into()), + }; + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "stale declaration is not a real regular file: {}", + target.display() + ); + remove_file_beneath(catalog, &target) + .with_context(|| format!("remove stale declaration {}", target.display()))?; + if is_canonical_agent_spec(&path) { + test_checkpoint("deleted-spec"); + } + test_checkpoint("mid-delete"); + } + for (path, desired_file) in &desired.files { + if current.is_some_and(|current| current.files.get(path) == Some(desired_file)) + || atomically_created + .iter() + .any(|prefix| path == prefix || path.starts_with(&format!("{prefix}/"))) + { + continue; + } + atomic_replace_file(control, catalog, path, desired_file)?; + test_checkpoint("mid-write"); + } + Ok(()) +} + +fn is_canonical_agent_spec(path: &str) -> bool { + let components = path.split('/').collect::>(); + components.len() == 4 && components[0] == "agents" && components[3] == "agent.kdl" +} + +fn create_new_identity_bundles( + control_file: &File, + catalog: &Path, + desired: &DeclarationProjection, +) -> Result> { + let mut created = BTreeSet::new(); + let control = retained_dir_path(control_file)?; + for path in desired.files.keys() { + let components = path.split('/').collect::>(); + if components.len() != 4 || components[0] != "agents" || components[3] != "agent.kdl" { + continue; + } + let prefix = components[..3].join("/"); + let target = catalog.join(&prefix); + match fs::symlink_metadata(&target) { + Ok(metadata) => { + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "canonical identity path is not a real directory: {}", + target.display() + ); + continue; + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + let host = target + .parent() + .context("identity target has no host parent")?; + ensure_real_dir_chain(catalog, host)?; + let stage = tempfile::Builder::new() + .prefix("catalog-apply-identity-") + .tempdir_in(&control)?; + fs::set_permissions(stage.path(), fs::Permissions::from_mode(0o755))?; + for (candidate, file) in &desired.files { + let Some(relative) = candidate.strip_prefix(&format!("{prefix}/")) else { + continue; + }; + let destination = stage.path().join(relative); + let parent = destination.parent().context("bundle file has no parent")?; + ensure_real_dir_chain(stage.path(), parent)?; + let mut output = OpenOptions::new() + .create_new(true) + .write(true) + .mode(if file.executable { 0o755 } else { 0o644 }) + .open(&destination)?; + output.write_all(&file.bytes)?; + output.sync_all()?; + } + sync_tree_dirs(stage.path())?; + test_checkpoint("identity-staged"); + test_forced_cross_device("identity-staged").map_err(control_plane_rename_error)?; + rename_noreplace_between_dirs(control_file, catalog, stage.path(), &target) + .map_err(control_plane_rename_error) + .with_context(|| format!("publish canonical identity bundle {}", target.display()))?; + sync_dir(host)?; + created.insert(prefix); + test_checkpoint("mid-write"); + } + Ok(created) +} + +fn atomic_replace_file( + control_file: &File, + catalog: &Path, + relative: &str, + file: &ProjectedFile, +) -> Result<()> { + let target = catalog.join(relative); + let parent = target + .parent() + .context("declaration target has no parent")?; + ensure_real_dir_chain(catalog, parent)?; + match fs::symlink_metadata(&target) { + Ok(metadata) => anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "declaration target is not a real regular file: {}", + target.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + let control = retained_dir_path(control_file)?; + let mut temp = tempfile::Builder::new() + .prefix("catalog-apply-leaf-") + .tempfile_in(&control)?; + temp.as_file_mut() + .set_permissions(fs::Permissions::from_mode(if file.executable { + 0o755 + } else { + 0o644 + }))?; + temp.write_all(&file.bytes)?; + temp.as_file().sync_all()?; + test_checkpoint("leaf-staged"); + test_forced_cross_device("leaf-staged").map_err(control_plane_rename_error)?; + persist_tempfile_from_control(control_file, catalog, temp, &target) + .map_err(control_plane_rename_error) + .with_context(|| format!("replace declaration {}", target.display()))?; + open_dir_beneath(catalog, parent)? + .sync_all() + .map_err(Into::into) +} + +fn remove_file_beneath(catalog: &Path, target: &Path) -> Result<()> { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + let parent = open_dir_beneath( + catalog, + target.parent().context("target has no parent directory")?, + )?; + let name = CString::new( + target + .file_name() + .context("target has no final component")? + .as_bytes(), + )?; + let result = unsafe { libc::unlinkat(parent.as_raw_fd(), name.as_ptr(), 0) }; + if result != 0 { + return Err(std::io::Error::last_os_error().into()); + } + parent.sync_all()?; + Ok(()) +} + +fn write_marker(control_file: &File, marker: &ApplyMarker) -> Result<()> { + let control = retained_dir_path(control_file)?; + let target = control.join(APPLY_MARKER); + let mut bytes = serde_json::to_vec(marker)?; + bytes.push(b'\n'); + let mut temp = tempfile::Builder::new() + .prefix(".catalog-apply-incomplete-") + .tempfile_in(&control)?; + temp.write_all(&bytes)?; + temp.as_file().sync_all()?; + fs::hard_link(temp.path(), &target) + .with_context(|| format!("publish catalog apply marker {}", target.display()))?; + temp.close()?; + control_file.sync_all().map_err(Into::into) +} + +fn read_marker_optional(path: &Path) -> Result> { + let mut file = match OpenOptions::new() + .read(true) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW) + .open(path) + { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error.into()), + }; + anyhow::ensure!( + file.metadata()?.is_file(), + "catalog apply marker is not a real regular file: {}", + path.display() + ); + let mut bytes = Vec::new(); + file.read_to_end(&mut bytes)?; + let marker = serde_json::from_slice(&bytes) + .with_context(|| format!("parse catalog apply marker {}", path.display()))?; + Ok(Some(marker)) +} + +fn validate_marker(marker: &ApplyMarker) -> Result<()> { + anyhow::ensure!( + marker.schema == MARKER_SCHEMA, + "unsupported catalog apply marker schema '{}'", + marker.schema + ); + validate_sha256(&marker.prepared_root_sha256)?; + validate_sha256(&marker.expected_root_sha256)?; + anyhow::ensure!( + marker + .original_paths + .windows(2) + .all(|pair| pair[0] < pair[1]), + "catalog apply marker originalPaths must be strictly sorted and unique" + ); + for path in &marker.original_paths { + validate_declaration_leaf_path(path)?; + } + let components = normal_components(Path::new(&marker.stage_name))?; + anyhow::ensure!( + components.len() == 1 && marker.stage_name == stage_name(&marker.prepared_root_sha256), + "catalog apply marker has an unsafe or non-canonical stage name" + ); + Ok(()) +} + +fn validate_declaration_leaf_path(path: &str) -> Result<()> { + let components = path.split('/').collect::>(); + anyhow::ensure!( + !components.is_empty() + && components + .iter() + .all(|component| !component.is_empty() && !matches!(*component, "." | "..")), + "catalog apply marker contains an unsafe declaration path" + ); + anyhow::ensure!( + !components + .iter() + .any(|component| matches!(*component, ".git" | ".st2")) + && components.first().copied() != Some("pty"), + "catalog apply marker contains a reserved declaration path" + ); + let canonical_bundle = components.len() >= 4 && components[0] == "agents"; + let template = components.len() >= 2 && components[0] == "_templates"; + anyhow::ensure!( + path == crate::catalog::CONFIG_FILE || canonical_bundle || template, + "catalog apply marker contains an unowned declaration path" + ); + if canonical_bundle { + anyhow::ensure!( + !matches!( + components[3], + ".workspace" | "resources" | "archive" | "inbox" | "status" + ) && !components[3].starts_with(".status.tmp-"), + "catalog apply marker contains a workspace or state-plane path" + ); + } + Ok(()) +} + +fn effective_pty_root(live_catalog: &Path, config: &crate::catalog::CatalogConfig) -> PathBuf { + match &config.pty_root { + Some(declared) => live_catalog.join(crate::expand::expand_catalog(declared, live_catalog)), + None => live_catalog.join("pty"), + } +} + +fn validate_live_workspace_facts(catalog: &Path, facts: &BTreeSet) -> Result<()> { + for relative in facts { + let path = catalog.join(relative); + ensure_real_dir_chain_present(catalog, &path, "live workspace fact")?; + } + Ok(()) +} + +fn validate_external_pty_root( + catalog: &Path, + config: &crate::catalog::CatalogConfig, +) -> Result<()> { + anyhow::ensure!( + config.pty_root.is_some(), + "catalog apply v1 requires an explicit external pty-root" + ); + let pty_root = lexical_absolute(&effective_pty_root(catalog, config))?; + anyhow::ensure!( + !pty_root.starts_with(catalog), + "catalog apply v1 requires pty-root outside the catalog: {}", + pty_root.display() + ); + Ok(()) +} + +fn lexical_absolute(path: &Path) -> Result { + anyhow::ensure!( + path.is_absolute(), + "path is not absolute: {}", + path.display() + ); + let mut normalized = PathBuf::from("/"); + for component in path.components() { + match component { + Component::RootDir | Component::CurDir => {} + Component::ParentDir => { + normalized.pop(); + } + Component::Normal(name) => normalized.push(name), + Component::Prefix(_) => anyhow::bail!("unsupported path prefix: {}", path.display()), + } + } + Ok(normalized) +} + +fn stage_name(root_sha256: &str) -> String { + format!("{STAGE_PREFIX}{root_sha256}") +} + +fn validate_sha256(value: &str) -> Result<()> { + anyhow::ensure!( + value.len() == 64 && value.bytes().all(|byte| byte.is_ascii_hexdigit()), + "expected sha256 must be exactly 64 hexadecimal characters" + ); + anyhow::ensure!( + value.bytes().all(|byte| !byte.is_ascii_uppercase()), + "expected sha256 must use lowercase hexadecimal" + ); + Ok(()) +} + +fn canonical_real_dir(path: &Path, label: &str) -> Result { + let canonical = path + .canonicalize() + .with_context(|| format!("canonicalize {label} {}", path.display()))?; + ensure_real_dir(&canonical, label)?; + Ok(canonical) +} + +fn canonical_real_dir_no_alias(path: &Path, label: &str) -> Result { + let metadata = fs::symlink_metadata(path) + .with_context(|| format!("inspect {label} {}", path.display()))?; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "{label} is not a real directory: {}", + path.display() + ); + canonical_real_dir(path, label) +} + +fn ensure_real_dir(path: &Path, label: &str) -> Result<()> { + let metadata = fs::symlink_metadata(path) + .with_context(|| format!("inspect {label} {}", path.display()))?; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "{label} is not a real directory: {}", + path.display() + ); + Ok(()) +} + +fn read_real_dir_optional(path: &Path) -> Result>> { + match fs::symlink_metadata(path) { + Ok(metadata) => anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "catalog path is not a real directory: {}", + path.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(error.into()), + } + Ok(Some(sorted_entries(path)?)) +} + +fn sorted_entries(path: &Path) -> Result> { + let mut entries = fs::read_dir(path) + .with_context(|| format!("read directory {}", path.display()))? + .collect::>>()?; + entries.sort_by_key(fs::DirEntry::file_name); + Ok(entries) +} + +fn ensure_safe_component(value: &std::ffi::OsStr, label: &str) -> Result<()> { + let value = value + .to_str() + .with_context(|| format!("{label} is not UTF-8"))?; + anyhow::ensure!( + !value.is_empty() && !matches!(value, "." | ".." | ".git" | ".st2"), + "{label} is not one safe path component" + ); + Ok(()) +} + +fn normalized_relative(root: &Path, path: &Path) -> Result { + let relative = path + .strip_prefix(root) + .with_context(|| format!("path escapes projection root: {}", path.display()))?; + let components = normal_components(relative)?; + anyhow::ensure!(!components.is_empty(), "projection cannot contain its root"); + Ok(components.join("/")) +} + +fn normal_components(path: &Path) -> Result> { + path.components() + .map(|component| match component { + Component::Normal(value) => value + .to_str() + .map(String::from) + .context("catalog path is not UTF-8"), + _ => anyhow::bail!("catalog path contains an unsafe component"), + }) + .collect() +} + +fn absolute_path(path: &Path) -> Result { + if path.is_absolute() { + Ok(path.to_path_buf()) + } else { + Ok(std::env::current_dir()?.join(path)) + } +} + +fn ensure_real_dir_chain(root: &Path, target: &Path) -> Result<()> { + let relative = target + .strip_prefix(root) + .context("directory escapes root")?; + let mut current = root.to_path_buf(); + for component in relative.components() { + let Component::Normal(name) = component else { + anyhow::bail!("directory contains an unsafe path component"); + }; + current.push(name); + match fs::symlink_metadata(¤t) { + Ok(metadata) => anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "path is not a real directory: {}", + current.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + fs::create_dir(¤t)?; + fs::set_permissions(¤t, fs::Permissions::from_mode(0o755))?; + sync_dir( + current + .parent() + .context("created directory has no parent")?, + )?; + } + Err(error) => return Err(error.into()), + } + } + Ok(()) +} + +fn ensure_real_dir_chain_present(root: &Path, target: &Path, label: &str) -> Result<()> { + let relative = target + .strip_prefix(root) + .with_context(|| format!("{label} escapes catalog root"))?; + let mut current = root.to_path_buf(); + for component in relative.components() { + let Component::Normal(name) = component else { + anyhow::bail!("{label} contains an unsafe path component"); + }; + current.push(name); + let metadata = fs::symlink_metadata(¤t) + .with_context(|| format!("inspect {label} {}", current.display()))?; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "{label} is not a real directory: {}", + current.display() + ); + } + Ok(()) +} + +fn sync_tree_dirs(root: &Path) -> Result<()> { + let mut dirs = Vec::new(); + collect_dirs(root, &mut dirs)?; + dirs.sort_by_key(|path| std::cmp::Reverse(path.components().count())); + for dir in dirs { + sync_dir(&dir)?; + } + Ok(()) +} + +fn collect_dirs(root: &Path, dirs: &mut Vec) -> Result<()> { + dirs.push(root.to_path_buf()); + for entry in sorted_entries(root)? { + let path = entry.path(); + let metadata = fs::symlink_metadata(&path)?; + if metadata.is_dir() && !metadata.file_type().is_symlink() { + collect_dirs(&path, dirs)?; + } + } + Ok(()) +} + +/// Copy an exact directory capability without following a source symlink at any depth. +pub(crate) fn capture_real_tree(source: &Path, destination: &Path) -> Result<()> { + capture_tree(source, destination, CaptureMode::General) +} + +fn capture_prepared_catalog(source: &Path, destination: &Path) -> Result<()> { + capture_tree(source, destination, CaptureMode::PreparedCatalog) +} + +#[derive(Clone, Copy, PartialEq, Eq)] +enum CaptureMode { + General, + PreparedCatalog, +} + +fn capture_tree(source: &Path, destination: &Path, mode: CaptureMode) -> Result<()> { + let source = OpenOptions::new() + .read(true) + .custom_flags(libc::O_CLOEXEC | libc::O_NOFOLLOW | libc::O_DIRECTORY) + .open(source) + .context("open retained source directory")?; + let source_root = retained_dir_path(&source)? + .canonicalize() + .context("resolve retained source directory")?; + let destination_root = destination + .canonicalize() + .context("resolve capture destination directory")?; + anyhow::ensure!( + !destination_root.starts_with(&source_root), + "capture destination {} is contained by source {}", + destination_root.display(), + source_root.display() + ); + capture_dir_capability(&source, destination, destination, mode)?; + sync_tree_dirs(destination) +} + +pub(crate) fn retained_dir_path(dir: &File) -> Result { + #[cfg(any(target_os = "linux", target_os = "android"))] + { + Ok(PathBuf::from(format!("/proc/self/fd/{}", dir.as_raw_fd()))) + } + #[cfg(target_os = "macos")] + { + Ok(PathBuf::from(format!("/dev/fd/{}", dir.as_raw_fd()))) + } + #[cfg(not(any(target_os = "linux", target_os = "android", target_os = "macos")))] + { + let _ = dir; + anyhow::bail!("retained-directory capture is unsupported on this platform") + } +} + +fn capture_dir_capability( + source: &File, + destination: &Path, + capture_root: &Path, + mode: CaptureMode, +) -> Result<()> { + for name in capability_dir_entries(source)? { + let mut input = openat_nofollow(source, &name)?; + let metadata = input.metadata()?; + let target = destination.join(&name); + if metadata.is_dir() { + let relative = target.strip_prefix(capture_root)?; + if mode == CaptureMode::PreparedCatalog && is_canonical_workspace_fact(relative) { + anyhow::ensure!( + capability_dir_entries(&input)?.is_empty(), + "prepared workspace fact must be empty: {}", + relative.display() + ); + fs::create_dir(&target)?; + fs::set_permissions(&target, fs::Permissions::from_mode(0o755))?; + continue; + } + fs::create_dir(&target)?; + fs::set_permissions(&target, fs::Permissions::from_mode(0o755))?; + capture_dir_capability(&input, &target, capture_root, mode)?; + } else if metadata.is_file() { + let relative = target.strip_prefix(capture_root)?; + if relative.components().next().and_then(|value| match value { + Component::Normal(name) => name.to_str(), + _ => None, + }) == Some("_templates") + { + anyhow::ensure!( + metadata.nlink() == 1, + "_templates contains a hard-linked file: {}", + target.display() + ); + } + let executable = metadata.permissions().mode() & 0o111 != 0; + let mut output = OpenOptions::new() + .create_new(true) + .write(true) + .mode(if executable { 0o755 } else { 0o644 }) + .open(&target)?; + std::io::copy(&mut input, &mut output)?; + output.sync_all()?; + } else { + anyhow::bail!( + "source tree contains a symlink or special entry: {}", + target.display() + ); + } + } + Ok(()) +} + +fn is_canonical_workspace_fact(path: &Path) -> bool { + let components = path.components().collect::>(); + components.len() == 4 + && matches!(components[0], Component::Normal(name) if name == "agents") + && matches!(components[3], Component::Normal(name) if name == ".workspace") +} + +fn openat_nofollow(parent: &File, name: &std::ffi::OsStr) -> Result { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + + let name = CString::new(name.as_bytes()).context("source entry name contains NUL")?; + let fd = unsafe { + libc::openat( + parent.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_CLOEXEC | libc::O_NOFOLLOW | libc::O_NONBLOCK, + ) + }; + if fd < 0 { + return Err(std::io::Error::last_os_error()).context("open retained source entry"); + } + Ok(unsafe { File::from_raw_fd(fd) }) +} + +fn capability_dir_entries(dir: &File) -> Result> { + let path = retained_dir_path(dir)?; + let mut names = fs::read_dir(&path)? + .map(|entry| entry.map(|entry| entry.file_name())) + .collect::>>()?; + names.sort(); + Ok(names) +} + +pub(crate) fn sync_dir(path: &Path) -> Result<()> { + File::open(path) + .with_context(|| format!("open directory {}", path.display()))? + .sync_all() + .with_context(|| format!("sync directory {}", path.display())) +} + +fn open_dir_nofollow(path: &Path) -> std::io::Result { + OpenOptions::new() + .read(true) + .custom_flags(libc::O_CLOEXEC | libc::O_DIRECTORY | libc::O_NOFOLLOW) + .open(path) +} + +pub(crate) fn open_dir_beneath(catalog: &Path, target: &Path) -> std::io::Result { + let relative = target.strip_prefix(catalog).map_err(|_| { + std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "directory escapes catalog", + ) + })?; + let mut current = open_dir_nofollow(catalog)?; + for component in relative.components() { + let Component::Normal(name) = component else { + return Err(std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "directory has an unsafe component", + )); + }; + current = openat_dir_nofollow(¤t, name)?; + } + Ok(current) +} + +pub(crate) fn openat_dir_nofollow(parent: &File, name: &std::ffi::OsStr) -> std::io::Result { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + let name = CString::new(name.as_bytes()).map_err(|_| { + std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "directory name contains NUL", + ) + })?; + let fd = unsafe { + libc::openat( + parent.as_raw_fd(), + name.as_ptr(), + libc::O_RDONLY | libc::O_CLOEXEC | libc::O_DIRECTORY | libc::O_NOFOLLOW, + ) + }; + if fd < 0 { + Err(std::io::Error::last_os_error()) + } else { + Ok(unsafe { File::from_raw_fd(fd) }) + } +} + +fn control_plane_rename_error(error: std::io::Error) -> anyhow::Error { + if error.raw_os_error() == Some(libc::EXDEV) { + anyhow::anyhow!( + "catalog control and declaration planes must share one filesystem for atomic publication" + ) + } else { + error.into() + } +} + +pub(crate) fn persist_tempfile_from_control( + control: &File, + catalog: &Path, + temp: tempfile::NamedTempFile, + target: &Path, +) -> std::io::Result<()> { + let source = temp.path(); + let target_parent = open_dir_beneath( + catalog, + target.parent().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no parent") + })?, + )?; + let source_name = source.file_name().ok_or_else(|| { + std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "temporary file has no name", + ) + })?; + let target_name = target.file_name().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no name") + })?; + renameat(control, source_name, &target_parent, target_name) +} + +pub(crate) fn link_tempfile_from_control( + control: &File, + catalog: &Path, + temp: &tempfile::NamedTempFile, + target: &Path, +) -> std::io::Result<()> { + let source = temp.path(); + let target_parent = open_dir_beneath( + catalog, + target.parent().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no parent") + })?, + )?; + let source_name = source.file_name().ok_or_else(|| { + std::io::Error::new( + std::io::ErrorKind::InvalidInput, + "temporary file has no name", + ) + })?; + let target_name = target.file_name().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no name") + })?; + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + let source_name = CString::new(source_name.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "source name contains NUL") + })?; + let target_name = CString::new(target_name.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target name contains NUL") + })?; + let result = unsafe { + libc::linkat( + control.as_raw_fd(), + source_name.as_ptr(), + target_parent.as_raw_fd(), + target_name.as_ptr(), + 0, + ) + }; + if result == 0 { + Ok(()) + } else { + Err(std::io::Error::last_os_error()) + } +} + +pub(crate) fn rename_noreplace_between_dirs( + control: &File, + catalog: &Path, + source: &Path, + target: &Path, +) -> std::io::Result<()> { + let target_parent = open_dir_beneath( + catalog, + target.parent().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no parent") + })?, + )?; + let source_name = source.file_name().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "source has no name") + })?; + let target_name = target.file_name().ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target has no name") + })?; + renameat_noreplace(control, source_name, &target_parent, target_name) +} + +fn renameat_noreplace( + source_parent: &File, + source: &std::ffi::OsStr, + target_parent: &File, + target: &std::ffi::OsStr, +) -> std::io::Result<()> { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + + let source = CString::new(source.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "source name contains NUL") + })?; + let target = CString::new(target.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target name contains NUL") + })?; + #[cfg(any(target_os = "linux", target_os = "android"))] + let result = unsafe { + libc::renameat2( + source_parent.as_raw_fd(), + source.as_ptr(), + target_parent.as_raw_fd(), + target.as_ptr(), + libc::RENAME_NOREPLACE, + ) + }; + #[cfg(target_os = "macos")] + let result = unsafe { + libc::renameatx_np( + source_parent.as_raw_fd(), + source.as_ptr(), + target_parent.as_raw_fd(), + target.as_ptr(), + libc::RENAME_EXCL, + ) + }; + #[cfg(not(any(target_os = "linux", target_os = "android", target_os = "macos")))] + let result = { + let _ = (source_parent, source, target_parent, target); + return Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + "atomic no-replace directory rename is unsupported on this platform", + )); + }; + if result == 0 { + Ok(()) + } else { + Err(std::io::Error::last_os_error()) + } +} + +fn renameat( + source_parent: &File, + source: &std::ffi::OsStr, + target_parent: &File, + target: &std::ffi::OsStr, +) -> std::io::Result<()> { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + + let source = CString::new(source.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "source name contains NUL") + })?; + let target = CString::new(target.as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target name contains NUL") + })?; + let result = unsafe { + libc::renameat( + source_parent.as_raw_fd(), + source.as_ptr(), + target_parent.as_raw_fd(), + target.as_ptr(), + ) + }; + if result == 0 { + Ok(()) + } else { + Err(std::io::Error::last_os_error()) + } +} + +pub(crate) fn rename_noreplace(source: &Path, target: &Path) -> std::io::Result<()> { + use std::ffi::CString; + use std::os::unix::ffi::OsStrExt as _; + + let source = CString::new(source.as_os_str().as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "source contains NUL") + })?; + let target = CString::new(target.as_os_str().as_bytes()).map_err(|_| { + std::io::Error::new(std::io::ErrorKind::InvalidInput, "target contains NUL") + })?; + #[cfg(any(target_os = "linux", target_os = "android"))] + let result = unsafe { + libc::renameat2( + libc::AT_FDCWD, + source.as_ptr(), + libc::AT_FDCWD, + target.as_ptr(), + libc::RENAME_NOREPLACE, + ) + }; + #[cfg(target_os = "macos")] + let result = unsafe { + libc::renameatx_np( + libc::AT_FDCWD, + source.as_ptr(), + libc::AT_FDCWD, + target.as_ptr(), + libc::RENAME_EXCL, + ) + }; + #[cfg(not(any(target_os = "linux", target_os = "android", target_os = "macos")))] + let result = { + let _ = (source, target); + return Err(std::io::Error::new( + std::io::ErrorKind::Unsupported, + "atomic no-replace directory rename is unsupported on this platform", + )); + }; + if result == 0 { + Ok(()) + } else { + Err(std::io::Error::last_os_error()) + } +} + +#[cfg(debug_assertions)] +fn test_checkpoint(point: &str) { + if std::env::var("ST2_TEST_CATALOG_APPLY_PAUSE_AT").as_deref() == Ok(point) + && let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_CATALOG_APPLY_READY"), + std::env::var("ST2_TEST_CATALOG_APPLY_RELEASE"), + ) + { + let _ = fs::write(&ready, point); + while !Path::new(&release).exists() { + std::thread::sleep(std::time::Duration::from_millis(2)); + } + } + if std::env::var("ST2_TEST_CATALOG_APPLY_CRASH_AT").as_deref() == Ok(point) { + std::process::abort(); + } +} + +#[cfg(not(debug_assertions))] +fn test_checkpoint(_point: &str) {} + +#[cfg(debug_assertions)] +fn test_forced_cross_device(point: &str) -> std::io::Result<()> { + if std::env::var("ST2_TEST_CATALOG_APPLY_EXDEV_AT").as_deref() == Ok(point) { + return Err(std::io::Error::from_raw_os_error(libc::EXDEV)); + } + Ok(()) +} + +#[cfg(not(debug_assertions))] +fn test_forced_cross_device(_point: &str) -> std::io::Result<()> { + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn workspace_directory_facts_are_typed_into_the_projection_hash() { + let files = BTreeMap::new(); + let no_facts = BTreeSet::new(); + let facts = BTreeSet::from(["agents/host/worker/.workspace".to_string()]); + assert_ne!( + hash_projection(&files, &no_facts), + hash_projection(&files, &facts) + ); + } +} diff --git a/src/context.rs b/src/context.rs index 46435607..a5f05764 100644 --- a/src/context.rs +++ b/src/context.rs @@ -93,6 +93,10 @@ pub fn write_now(context_dir: &Path, content: &str) -> anyhow::Result<()> { /// scannable list; multi-line reasoning belongs in a doc). Renders `- . why: .` /// into a fresh `decisions/-.md`. Returns the entry's filename. pub fn append_decision(context_dir: &Path, decision: &str, why: &str) -> anyhow::Result { + append_decision_to_dir(&decisions_dir(context_dir), decision, why) +} + +pub fn append_decision_to_dir(dir: &Path, decision: &str, why: &str) -> anyhow::Result { let decision = decision.trim(); let why = why.trim(); if decision.is_empty() { @@ -106,8 +110,7 @@ pub fn append_decision(context_dir: &Path, decision: &str, why: &str) -> anyhow: "context append: --decision and --why must be single lines (the log is a scannable list)" ); } - let dir = decisions_dir(context_dir); - fs::create_dir_all(&dir)?; + fs::create_dir_all(dir)?; let line = format!( "- {} {}. why: {}.\n", iso_utc_now(), diff --git a/src/expand.rs b/src/expand.rs index acb137f5..cc3027ba 100644 --- a/src/expand.rs +++ b/src/expand.rs @@ -8,11 +8,20 @@ //! literal token (`$VAR`) rather than blanked — a spawn-time path with an undefined var is a //! misconfiguration worth seeing, not one to silently turn into a wrong absolute path. +use std::path::{Component, Path, PathBuf}; + +use anyhow::Result; + /// Expand `$VAR` / `${VAR}` / `$$` in `input`, resolving names via `lookup`. Unknown names are left /// as their literal token. pub fn expand_vars(input: &str, lookup: impl Fn(&str) -> Option) -> String { + expand_vars_checked(input, lookup).0 +} + +fn expand_vars_checked(input: &str, lookup: impl Fn(&str) -> Option) -> (String, bool) { let mut out = String::with_capacity(input.len()); let mut chars = input.chars().peekable(); + let mut unresolved = false; while let Some(c) = chars.next() { if c != '$' { @@ -39,6 +48,7 @@ pub fn expand_vars(input: &str, lookup: impl Fn(&str) -> Option) -> Stri match (closed, lookup(&name)) { (true, Some(v)) => out.push_str(&v), (true, None) => { + unresolved = true; out.push_str("${"); out.push_str(&name); out.push('}'); @@ -63,6 +73,7 @@ pub fn expand_vars(input: &str, lookup: impl Fn(&str) -> Option) -> Stri match lookup(&name) { Some(v) => out.push_str(&v), None => { + unresolved = true; out.push('$'); out.push_str(&name); } @@ -72,7 +83,7 @@ pub fn expand_vars(input: &str, lookup: impl Fn(&str) -> Option) -> Stri _ => out.push('$'), } } - out + (out, unresolved) } /// Expand against st2's ambient process environment (the production lookup). @@ -84,10 +95,65 @@ pub fn expand_env(input: &str) -> String { pub fn expand_catalog(input: &str, catalog_root: &std::path::Path) -> String { let catalog = catalog_root.display().to_string(); expand_vars(input, move |k| { - if k == "CATALOG" { Some(catalog.clone()) } else { std::env::var(k).ok() } + if k == "CATALOG" { + Some(catalog.clone()) + } else { + std::env::var(k).ok() + } }) } +/// Resolve one authored workspace/cwd exactly as launch does, then classify its relative grammar. +/// +/// Absolute values remain valid external or catalog paths. A relative value is valid only when +/// expansion and lexical normalization select the declaring bundle's `.workspace`. Unknown +/// variables fail closed instead of silently evading catalog ownership classification. +pub fn resolve_spec_path(raw: &str, catalog_root: &Path, spec_dir: &Path) -> Result { + let catalog = catalog_root.display().to_string(); + let (expanded, unresolved) = expand_vars_checked(raw, move |key| { + if key == "CATALOG" { + Some(catalog.clone()) + } else { + std::env::var(key).ok() + } + }); + anyhow::ensure!( + !unresolved, + "path has an unresolved environment variable: {raw}" + ); + + let expanded = Path::new(&expanded); + let resolved = lexical_absolute(&spec_dir.join(expanded))?; + if expanded.is_relative() { + let canonical_workspace = lexical_absolute(&spec_dir.join(".workspace"))?; + anyhow::ensure!( + resolved == canonical_workspace, + "relative path must resolve to canonical '.workspace': {raw}" + ); + } + Ok(resolved) +} + +fn lexical_absolute(path: &Path) -> Result { + anyhow::ensure!( + path.is_absolute(), + "path is not absolute: {}", + path.display() + ); + let mut normalized = PathBuf::from("/"); + for component in path.components() { + match component { + Component::RootDir | Component::CurDir => {} + Component::ParentDir => { + normalized.pop(); + } + Component::Normal(name) => normalized.push(name), + Component::Prefix(_) => anyhow::bail!("unsupported path prefix: {}", path.display()), + } + } + Ok(normalized) +} + #[cfg(test)] mod tests { use super::*; @@ -101,8 +167,14 @@ mod tests { fn expands_bare_and_braced_vars() { let env = HashMap::from([("TEAM_ROOT", "/net/foo")]); assert_eq!(expand_vars("$TEAM_ROOT/bus", lookup(&env)), "/net/foo/bus"); - assert_eq!(expand_vars("${TEAM_ROOT}/pty", lookup(&env)), "/net/foo/pty"); - assert_eq!(expand_vars("pre-${TEAM_ROOT}-post", lookup(&env)), "pre-/net/foo-post"); + assert_eq!( + expand_vars("${TEAM_ROOT}/pty", lookup(&env)), + "/net/foo/pty" + ); + assert_eq!( + expand_vars("pre-${TEAM_ROOT}-post", lookup(&env)), + "pre-/net/foo-post" + ); } #[test] @@ -131,7 +203,10 @@ mod tests { let env: HashMap<&str, &str> = HashMap::new(); assert_eq!(expand_vars("a$/b", lookup(&env)), "a$/b"); assert_eq!(expand_vars("trailing$", lookup(&env)), "trailing$"); - assert_eq!(expand_vars("unterminated ${FOO", lookup(&env)), "unterminated ${FOO"); + assert_eq!( + expand_vars("unterminated ${FOO", lookup(&env)), + "unterminated ${FOO" + ); } #[test] diff --git a/src/lib.rs b/src/lib.rs index c505662e..3f5d2dbd 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -9,6 +9,7 @@ pub mod agent_publish; pub mod agents; pub mod catalog; pub mod catalog_lock; +pub mod catalog_transaction; pub mod context; pub mod ding; pub mod eval_run; diff --git a/src/main.rs b/src/main.rs index f153a03c..f6bd5041 100644 --- a/src/main.rs +++ b/src/main.rs @@ -127,6 +127,9 @@ enum Command { /// Transactionally publish one canonical Agent Spec into the live catalog. #[command(subcommand)] Agent(AgentCmd), + /// Canonical declaration snapshots and crash-recoverable whole-catalog application. + #[command(subcommand)] + Catalog(CatalogCmd), /// Explicit teardown: kill every live task of this host's catalog agents. The ONLY thing that ends /// tasks (stopping/crashing st2 never does). Idempotent. Down { @@ -262,6 +265,28 @@ enum Command { #[derive(Subcommand)] enum AgentCmd { + /// Compute the authoritative digest bound by `agent publish --input-sha256`. + Digest { + /// A canonical KDL file containing exactly one top-level `agent` node. + #[arg( + long, + value_name = "FILE", + required_unless_present = "bundle", + conflicts_with = "bundle" + )] + spec: Option, + /// A create-only directory whose root contains exactly one canonical `agent.kdl`. + #[arg( + long, + value_name = "DIR", + required_unless_present = "spec", + conflicts_with = "spec" + )] + bundle: Option, + /// Emit the typed source-digest receipt as JSON. + #[arg(long)] + json: bool, + }, /// Publish exactly one explicit-host, explicit-identity agent under a catalog-wide CAS lock. Publish { /// A canonical KDL file containing exactly one top-level `agent` node. @@ -295,12 +320,53 @@ enum AgentCmd { conflicts_with = "expect_absent" )] expect_sha256: Option, + /// SHA-256 returned by `st2 agent digest` for the exact source capability. + #[arg(long, value_name = "HEX")] + input_sha256: String, /// Emit the typed publication result as JSON. #[arg(long)] json: bool, }, } +#[derive(Subcommand)] +enum CatalogCmd { + /// Capture the coherent declaration plane into a create-only canonical directory. + Snapshot { + /// Destination directory. It must be outside the live catalog. + #[arg(long, value_name = "DIR")] + output: PathBuf, + /// Emit the typed snapshot receipt as JSON. + #[arg(long)] + json: bool, + }, + /// Apply a complete canonical declaration directory under declaration-root CAS. + Apply { + /// Complete prepared declaration directory. Runtime state and control paths are rejected. + #[arg( + long, + value_name = "DIR", + required_unless_present = "resume", + conflicts_with = "resume" + )] + prepared: Option, + /// Expected canonical declaration-root SHA-256 of the live catalog. + #[arg( + long, + value_name = "HEX", + required_unless_present = "resume", + conflicts_with = "resume" + )] + expect_sha256: Option, + /// Resume the durable incomplete marker and internal stage without the original source. + #[arg(long, conflicts_with_all = ["prepared", "expect_sha256"])] + resume: bool, + /// Emit the typed application receipt as JSON. + #[arg(long)] + json: bool, + }, +} + /// Shared context for message subcommands: where the catalog is, who "I" am, and the local host. /// Defaults come from the same env st2 sets on every task it spawns (`$CATALOG`, `$ST_AGENT`), so a /// running agent needs no flags. @@ -619,6 +685,7 @@ fn main() -> Result<()> { bundle, expect_absent, expect_sha256, + input_sha256, json, }) => { let catalog = catalog_arg(None)?; @@ -636,6 +703,7 @@ fn main() -> Result<()> { catalog, source, expectation, + input_sha256, })?; if json { println!("{}", serde_json::to_string_pretty(&result)?); @@ -652,6 +720,76 @@ fn main() -> Result<()> { } Ok(()) } + Command::Agent(AgentCmd::Digest { spec, bundle, json }) => { + let source = match (spec, bundle) { + (Some(path), None) => st2::agent_publish::PublishSource::Spec(path), + (None, Some(path)) => st2::agent_publish::PublishSource::Bundle(path), + _ => unreachable!("clap enforces one source"), + }; + let digest = st2::agent_publish::digest_source(source)?; + if json { + println!("{}", serde_json::to_string_pretty(&digest)?); + } else { + println!("{}", digest.sha256); + } + Ok(()) + } + Command::Catalog(CatalogCmd::Snapshot { output, json }) => { + let result = + st2::catalog_transaction::snapshot(st2::catalog_transaction::SnapshotRequest { + catalog: catalog_arg(None)?, + output, + })?; + if json { + println!("{}", serde_json::to_string_pretty(&result)?); + } else { + println!( + "{} {} {}", + match result.status { + st2::catalog_transaction::SnapshotStatus::Created => "created", + st2::catalog_transaction::SnapshotStatus::Unchanged => "unchanged", + }, + result.root_sha256, + result.output.display() + ); + } + Ok(()) + } + Command::Catalog(CatalogCmd::Apply { + prepared, + expect_sha256, + resume, + json, + }) => { + let mode = if resume { + st2::catalog_transaction::ApplyMode::Resume + } else { + let prepared = prepared.context("clap requires --prepared unless --resume")?; + let expect_sha256 = + expect_sha256.context("clap requires --expect-sha256 unless --resume")?; + st2::catalog_transaction::ApplyMode::Prepared { + prepared, + expect_sha256, + } + }; + let result = st2::catalog_transaction::apply(st2::catalog_transaction::ApplyRequest { + catalog: catalog_arg(None)?, + mode, + })?; + if json { + println!("{}", serde_json::to_string_pretty(&result)?); + } else { + println!( + "{} {}", + match result.status { + st2::catalog_transaction::ApplyStatus::Applied => "applied", + st2::catalog_transaction::ApplyStatus::Unchanged => "unchanged", + }, + result.after_sha256 + ); + } + Ok(()) + } Command::Agents { catalog, status, @@ -1168,13 +1306,33 @@ fn tasks_cmd(root: &Path, host: Option) -> Result<()> { anyhow::bail!("task inventory incomplete") } }; + let before = match st2::catalog_lock::read_fence(&catalog) { + Ok(fence) => fence, + Err(error) => return print_incomplete_tasks(catalog, host, error.to_string()), + }; let found = discover(&catalog); + let observed = match st2::catalog_lock::read_fence(&catalog) { + Ok(fence) if fence == before => fence, + Ok(_) => { + return print_incomplete_tasks( + catalog, + host, + "catalog generation changed during task discovery".to_string(), + ); + } + Err(error) => return print_incomplete_tasks(catalog, host, error.to_string()), + }; let runner = SystemRunner::new(catalog.clone(), exec_state_dir(&host)); let mut inventory = st2::task_inventory::inventory(&catalog, &host, &found, &runner); let after = discover(&catalog); if !st2::task_inventory::same_discovery(&found, &after) { inventory.mark_incomplete("catalog declarations changed during task observation"); } + match st2::catalog_lock::read_fence(&catalog) { + Ok(after) if after == observed => {} + Ok(_) => inventory.mark_incomplete("catalog generation changed during task observation"), + Err(error) => inventory.mark_incomplete(error.to_string()), + } println!("{}", inventory.to_json()); if inventory.complete() { Ok(()) @@ -1183,6 +1341,12 @@ fn tasks_cmd(root: &Path, host: Option) -> Result<()> { } } +fn print_incomplete_tasks(catalog: PathBuf, host: String, detail: String) -> Result<()> { + let inventory = st2::task_inventory::TaskInventory::incomplete(catalog, host, detail); + println!("{}", inventory.to_json()); + anyhow::bail!("task inventory incomplete") +} + fn tool_on_path(tool: &str) -> bool { std::env::var_os("PATH") .map(|p| std::env::split_paths(&p).any(|d| d.join(tool).is_file())) @@ -1278,7 +1442,9 @@ fn status_cmd(identity: Option, set: Option, ctx: MsgCtx) -> Res let state = st2::status::State::parse_settable(&word).with_context(|| { format!("invalid state '{word}' (settable: offline|available|busy|away|dnd)") })?; - st2::status::set_state(&sp, state)?; + message::with_resolved_agent_dir(&root, &id, &host, |agent| { + st2::status::set_state(&st2::status::status_path(agent), state) + })?; println!("status: {}", state.as_str()); } } @@ -1344,7 +1510,7 @@ fn ding_cmd( let session = session.unwrap_or_else(|| id.clone()); // Flat-bus aware: a native catalog agent → its resources/inbox; a catalog-LESS bus (an eval's // ST_ROOT) → the flat //inbox. Status lives beside it either way. - let agent_dir = message::resolve_agent_dir(&catalog_root, &id, &this_host) + let agent_dir = message::resolve_agent_dir(&catalog_root, &id, &this_host)? .unwrap_or_else(|| catalog_root.join(&id)); let inbox = resolve_message_inbox(&catalog_root, &id, &this_host)?; let status_path = st2::status::status_path(&agent_dir); @@ -1381,7 +1547,7 @@ fn acting_id(ctx: &MsgCtx) -> Result { /// Resolve a recipient/identity to its agent folder in the catalog, or a clear error. fn agent_dir_of(root: &Path, id: &str, host: &str) -> Result { - message::resolve_agent_dir(root, id, host) + message::resolve_agent_dir(root, id, host)? .with_context(|| format!("no agent '{id}' found in catalog {}", root.display())) } @@ -1432,9 +1598,10 @@ fn message_cmd(cmd: MessageCmd) -> Result<()> { let (root, host) = resolve_ctx(&ctx)?; let from = acting_id(&ctx)?; let body = body_or_stdin(body)?; - let dir = resolve_message_inbox(&root, &to, &host)?; - let filename = message::send_to_inbox( - &dir, + let filename = send_resolved_message( + &root, + &to, + &host, &from, subject.as_deref(), in_reply_to.as_deref(), @@ -1461,9 +1628,10 @@ fn message_cmd(cmd: MessageCmd) -> Result<()> { .with_context(|| format!("message '{filename}' has no `from` to reply to"))?; let subject = subject.or_else(|| message::reply_subject(original.subject.as_deref())); let body = body_or_stdin(body)?; - let dir = resolve_message_inbox(&root, &to, &host)?; - let sent = message::send_to_inbox( - &dir, + let sent = send_resolved_message( + &root, + &to, + &host, &from, subject.as_deref(), Some(&filename), @@ -1567,9 +1735,7 @@ fn message_cmd(cmd: MessageCmd) -> Result<()> { MessageCmd::Archive { first, second, ctx } => { let (root, host) = resolve_ctx(&ctx)?; let (id, filename) = box_target(first, second, &ctx)?; - let inbox = resolve_message_inbox(&root, &id, &host)?; - let archive = message::resolve_archive(&root, &id, &host)?; - message::archive_msg(&inbox, &archive, &filename)?; + message::archive_resolved_message(&root, &id, &host, &filename)?; println!("archived"); Ok(()) } @@ -1582,7 +1748,7 @@ fn message_cmd(cmd: MessageCmd) -> Result<()> { let (root, _host) = resolve_ctx(&ctx)?; // `[identity] filename` — the identity is irrelevant (the walk is catalog-wide). let filename = second.unwrap_or(first); - let mut entries = message::collect_thread(&root, &filename); + let mut entries = message::collect_thread(&root, &filename)?; if entries.is_empty() { anyhow::bail!( "no thread found for '{filename}' in catalog {}", @@ -1607,6 +1773,24 @@ fn message_cmd(cmd: MessageCmd) -> Result<()> { } } +fn send_resolved_message( + root: &Path, + to: &str, + host: &str, + from: &str, + subject: Option<&str>, + in_reply_to: Option<&str>, + tags: &[String], + body: &str, +) -> Result { + if std::env::var("ST2_EVAL_REQUESTER").as_deref() == Ok(to) { + let inbox = resolve_message_inbox(root, to, host)?; + message::send_to_inbox(&inbox, from, subject, in_reply_to, tags, body) + } else { + message::send_to_resolved_inbox(root, to, host, from, subject, in_reply_to, tags, body) + } +} + /// `st2 message ls --json` row (stable st2 wire contract). #[derive(serde::Serialize)] struct LsItemJson<'a> { @@ -1705,10 +1889,21 @@ fn context_cmd(cmd: ContextCmd) -> Result<()> { Ok(()) } ContextCmd::Write { identity, ctx } => { - let dir = resolve_context_dir(identity, &ctx)?; + let (root, host) = resolve_ctx(&ctx)?; + let id = match identity { + Some(identity) => identity, + None => acting_id(&ctx)?, + }; let content = std::io::read_to_string(std::io::stdin()).context("reading context from stdin")?; - context::write_now(&dir, &content)?; + message::with_resolved_state_dir( + &root, + &id, + &host, + &["resources", "context"], + true, + |dir| context::write_now(dir, &content), + )?; eprintln!("context: wrote now.md ({} bytes)", content.len()); Ok(()) } @@ -1718,8 +1913,19 @@ fn context_cmd(cmd: ContextCmd) -> Result<()> { why, ctx, } => { - let dir = resolve_context_dir(identity, &ctx)?; - let filename = context::append_decision(&dir, &decision, &why)?; + let (root, host) = resolve_ctx(&ctx)?; + let id = match identity { + Some(identity) => identity, + None => acting_id(&ctx)?, + }; + let filename = message::with_resolved_state_dir( + &root, + &id, + &host, + &["resources", "context", "decisions"], + true, + |dir| context::append_decision_to_dir(dir, &decision, &why), + )?; println!("{filename}"); Ok(()) } @@ -1756,19 +1962,28 @@ fn resource_cmd(cmd: ResourceCmd) -> Result<()> { ctx, } => { let (root, host) = resolve_ctx(&ctx)?; - let dir = st2::resource::links_dir(&agent_dir_of(&root, &acting_id(&ctx)?, &host)?); + let id = acting_id(&ctx)?; let body = if body_stdin { std::io::read_to_string(std::io::stdin())? } else { String::new() }; - let f = st2::resource::add( - &dir, - &url, - title.as_deref(), - &tags, - relation.as_deref(), - &body, + let f = message::with_resolved_state_dir( + &root, + &id, + &host, + &["resources", "links"], + true, + |dir| { + st2::resource::add( + dir, + &url, + title.as_deref(), + &tags, + relation.as_deref(), + &body, + ) + }, )?; println!("{f}"); Ok(()) @@ -1812,9 +2027,17 @@ fn resource_cmd(cmd: ResourceCmd) -> Result<()> { ResourceCmd::Remove { first, second, ctx } => { let (root, host) = resolve_ctx(&ctx)?; let (id, filename) = box_target(first, second, &ctx)?; - st2::resource::remove( - &st2::resource::links_dir(&agent_dir_of(&root, &id, &host)?), - &filename, + anyhow::ensure!( + message::is_message_filename(&filename), + "invalid resource filename {filename:?}" + ); + message::with_resolved_state_dir( + &root, + &id, + &host, + &["resources", "links"], + false, + |dir| st2::resource::remove(dir, &filename), )?; println!("removed"); Ok(()) diff --git a/src/materialize.rs b/src/materialize.rs index 4001aa1a..140b21c6 100644 --- a/src/materialize.rs +++ b/src/materialize.rs @@ -48,10 +48,8 @@ fn references_variable(input: &str, variable: &str) -> bool { while input.contains(&marker) { marker.push('\0'); } - crate::expand::expand_vars(input, |name| { - (name == variable).then(|| marker.clone()) - }) - .contains(&marker) + crate::expand::expand_vars(input, |name| (name == variable).then(|| marker.clone())) + .contains(&marker) } impl RenderOp { @@ -61,8 +59,7 @@ impl RenderOp { source, destination, } => { - references_variable(source, variable) - || references_variable(destination, variable) + references_variable(source, variable) || references_variable(destination, variable) } Self::File { destination, @@ -72,12 +69,10 @@ impl RenderOp { destination, content, } => { - references_variable(destination, variable) - || references_variable(content, variable) + references_variable(destination, variable) || references_variable(content, variable) } Self::EnsureLine { destination, line } => { - references_variable(destination, variable) - || references_variable(line, variable) + references_variable(destination, variable) || references_variable(line, variable) } Self::GitExclude { path } => references_variable(path, variable), } @@ -240,6 +235,35 @@ pub fn parse_plan(spec: &AgentSpec) -> Result { } } +/// Catalog-owned files read by this agent's `render { copy ... }` operations. +/// +/// Absolute/external sources are deliberately absent: a declaration snapshot owns catalog bytes, +/// not arbitrary workspace or host files. The returned paths are exact existing files; callers +/// still decide which file kinds are admissible for their transaction. +pub(crate) fn catalog_owned_render_inputs( + root: &Path, + spec: &AgentSpec, + this_host: &str, +) -> Result> { + let plan = parse_plan(spec)?; + let env = render_env(root, spec, this_host); + let spec_dir = spec.path.parent().unwrap_or(root); + let mut inputs = BTreeSet::new(); + for operation in plan.ops { + let RenderOp::Copy { + source: raw_source, .. + } = operation + else { + continue; + }; + let resolved = source(root, spec_dir, &raw_source, &env)?; + if resolved.strip_prefix(root).is_ok() { + inputs.insert(resolved); + } + } + Ok(inputs.into_iter().collect()) +} + fn render_env(root: &Path, spec: &AgentSpec, this_host: &str) -> BTreeMap { let mut env = BTreeMap::from([ ("CATALOG".to_string(), root.display().to_string()), diff --git a/src/message.rs b/src/message.rs index f6520a4a..573ea878 100644 --- a/src/message.rs +++ b/src/message.rs @@ -12,12 +12,14 @@ //! the catalog for VRS-native, or `$ST_ROOT` for a compat shim). use std::collections::{HashMap, HashSet}; -use std::fs; +use std::fs::{self, File, OpenOptions}; use std::io::Read; use std::path::{Component, Path, PathBuf}; use std::sync::atomic::{AtomicU64, Ordering}; use std::time::{SystemTime, UNIX_EPOCH}; +use anyhow::Context as _; + /// The alphabet st2 *generates* `` from — Crockford base32 (`0-9a-z` minus `i l o u`). This is /// a strict subset of what the reader accepts: the frozen bus grammar is `[0-9a-z]{6}`, so a peer /// may legally use i/l/o/u and [`is_message_filename`] must not reject those. @@ -328,7 +330,10 @@ impl ExternalInbox { let mut components = Path::new(identity).components(); let safe = matches!(components.next(), Some(Component::Normal(component)) if component == identity) && components.next().is_none(); - anyhow::ensure!(safe, "external requester identity must be one non-empty relative path component"); + anyhow::ensure!( + safe, + "external requester identity must be one non-empty relative path component" + ); Ok(Self { root: root.to_path_buf(), identity: identity.to_owned(), @@ -365,9 +370,7 @@ pub fn resolve_inbox_with_external( Ok(inbox) => Ok(inbox), Err(error) => match external { Some(external) - if external.root == root - && external.identity == id - && external.inbox.is_dir() => + if external.root == root && external.identity == id && external.inbox.is_dir() => { Ok(external.inbox.clone()) } @@ -400,6 +403,19 @@ pub fn resolve_list_box( if orphan { return Ok(flat()); } + if apply_incomplete(root) { + return resolve_agent_dir(root, id, host)? + .map(|agent_dir| { + if archive { + archive_dir(&agent_dir) + } else { + inbox_dir(&agent_dir) + } + }) + .with_context(|| { + format!("agent '{id}' is not addressable while catalog apply is incomplete") + }); + } let discovered = crate::discover(root); if let Some(agent_dir) = discovered .specs @@ -422,13 +438,292 @@ pub fn resolve_list_box( /// Resolve a recipient (a bus id `.` or a bare identity) to its agent folder in the /// catalog, via content discovery. Returns `None` if no agent matches. -pub fn resolve_agent_dir(catalog_root: &Path, recipient: &str, this_host: &str) -> Option { - let found = crate::discover(catalog_root); - found - .specs - .into_iter() - .find(|s| s.bus_id(this_host) == recipient || s.identity == recipient) - .and_then(|s| s.path.parent().map(Path::to_path_buf)) +pub fn resolve_agent_dir( + catalog_root: &Path, + recipient: &str, + this_host: &str, +) -> anyhow::Result> { + Ok(resolve_agent_handle(catalog_root, recipient, this_host)?.map(|agent| agent.path)) +} + +pub fn with_resolved_agent_dir( + catalog_root: &Path, + identity: &str, + this_host: &str, + operation: impl FnOnce(&Path) -> anyhow::Result, +) -> anyhow::Result { + with_resolved_state_dir(catalog_root, identity, this_host, &[], true, operation) +} + +pub fn with_resolved_state_dir( + catalog_root: &Path, + identity: &str, + this_host: &str, + components: &[&str], + create: bool, + operation: impl FnOnce(&Path) -> anyhow::Result, +) -> anyhow::Result { + match resolve_agent_handle(catalog_root, identity, this_host)? { + Some(agent) => { + test_capability_checkpoint(); + let path = match agent.capability.as_ref() { + Some(capability) if components.is_empty() => { + crate::catalog_transaction::retained_dir_path(capability)? + } + Some(capability) => { + let directory = open_message_box(capability, components, create)? + .context("resolved state directory does not exist")?; + let path = crate::catalog_transaction::retained_dir_path(&directory)?; + return operation(&path); + } + None => components + .iter() + .fold(agent.path, |path, component| path.join(component)), + }; + operation(&path) + } + None => { + let discovered = crate::discover(catalog_root); + anyhow::ensure!( + crate::catalog_transaction::catalog_transition(catalog_root)?.is_none() + && !catalog_root.join(crate::catalog_lock::CONTROL_DIR).exists() + && discovered.specs.is_empty() + && discovered.errors.is_empty(), + "no agent '{identity}' found in catalog {}", + catalog_root.display() + ); + operation( + &components + .iter() + .fold(catalog_root.join(identity), |path, component| { + path.join(component) + }), + ) + } + } +} + +fn resolve_agent_handle( + catalog_root: &Path, + recipient: &str, + this_host: &str, +) -> anyhow::Result> { + for _ in 0..3 { + let before = address_fence(catalog_root)?; + let mut candidates = addressable_agent_dirs(catalog_root, this_host, before.1.as_ref())? + .into_iter() + .filter(|candidate| candidate.bus_id == recipient || candidate.identity == recipient) + .collect::>(); + let after = address_fence(catalog_root)?; + if before != after { + continue; + } + candidates.sort_by(|left, right| left.path.cmp(&right.path)); + candidates.dedup_by(|left, right| left.path == right.path); + return Ok((candidates.len() == 1).then(|| candidates.remove(0))); + } + anyhow::bail!("catalog address book changed repeatedly while resolving {recipient:?}") +} + +fn address_fence( + catalog_root: &Path, +) -> anyhow::Result<( + Option, + Option, +)> { + let first_generation = crate::catalog_lock::read_generation_token(catalog_root)?; + let transition = crate::catalog_transaction::catalog_transition(catalog_root)?; + test_address_fence_checkpoint(); + let second_generation = crate::catalog_lock::read_generation_token(catalog_root)?; + anyhow::ensure!( + first_generation == second_generation, + "catalog address book changed while sampling its transition fence" + ); + Ok((second_generation, transition)) +} + +#[cfg(debug_assertions)] +fn test_address_fence_checkpoint() { + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_ADDRESS_FENCE_READY"), + std::env::var("ST2_TEST_ADDRESS_FENCE_RELEASE"), + ) else { + return; + }; + if OpenOptions::new() + .write(true) + .create_new(true) + .open(&ready) + .is_err() + { + return; + } + while !Path::new(&release).exists() { + std::thread::yield_now(); + } +} + +#[cfg(not(debug_assertions))] +fn test_address_fence_checkpoint() {} + +#[derive(Debug)] +struct AddressableAgent { + bus_id: String, + identity: String, + path: PathBuf, + capability: Option, +} + +fn addressable_agent_dirs( + catalog_root: &Path, + this_host: &str, + transition: Option<&crate::catalog_transaction::CatalogTransition>, +) -> anyhow::Result> { + let Some(transition) = transition else { + return crate::discover(catalog_root) + .specs + .into_iter() + .map(|spec| { + let path = spec + .path + .parent() + .context("Agent Spec has no identity directory")? + .to_path_buf(); + let capability = crate::catalog_transaction::open_dir_beneath(catalog_root, &path)?; + Ok(AddressableAgent { + bus_id: spec.bus_id(this_host), + identity: spec.identity, + path, + capability: Some(capability), + }) + }) + .collect(); + }; + let agents = catalog_root.join("agents"); + let metadata = match fs::symlink_metadata(&agents) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()), + Err(error) => return Err(error.into()), + }; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "canonical agents path is not a real directory: {}", + agents.display() + ); + let mut result = Vec::new(); + for host in sorted_real_entries(&agents, "host")? { + let host_name = safe_entry_name(&host, "host")?; + for identity in sorted_real_entries(&host.path(), "identity")? { + let identity_name = safe_entry_name(&identity, "identity")?; + let key = crate::catalog_transaction::AgentKey { + host: host_name.clone(), + identity: identity_name.clone(), + }; + let path = identity.path(); + let capability = crate::catalog_transaction::open_dir_beneath(catalog_root, &path)?; + let retained = crate::catalog_transaction::retained_dir_path(&capability)?; + let current_spec = marker_spec_matches(&retained, &key)?; + let retained_state = + transition.original_agents.contains(&key) && marker_state_exists(&retained)?; + if current_spec || retained_state { + result.push(AddressableAgent { + bus_id: format!("{}.{}", key.host, key.identity), + identity: key.identity, + path, + capability: Some(capability), + }); + } + } + } + Ok(result) +} + +fn sorted_real_entries(dir: &Path, label: &str) -> anyhow::Result> { + let mut entries = fs::read_dir(dir)?.collect::>>()?; + entries.sort_by_key(fs::DirEntry::file_name); + for entry in &entries { + let metadata = fs::symlink_metadata(entry.path())?; + anyhow::ensure!( + metadata.is_dir() && !metadata.file_type().is_symlink(), + "canonical {label} path is not a real directory: {}", + entry.path().display() + ); + } + Ok(entries) +} + +fn safe_entry_name(entry: &fs::DirEntry, label: &str) -> anyhow::Result { + let value = entry + .file_name() + .into_string() + .map_err(|_| anyhow::anyhow!("canonical {label} is not UTF-8"))?; + anyhow::ensure!(safe_component(&value), "unsafe canonical {label} {value:?}"); + Ok(value) +} + +fn marker_spec_matches( + agent_dir: &Path, + key: &crate::catalog_transaction::AgentKey, +) -> anyhow::Result { + let path = agent_dir.join("agent.kdl"); + let metadata = match fs::symlink_metadata(&path) { + Ok(metadata) => metadata, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(false), + Err(error) => return Err(error.into()), + }; + anyhow::ensure!( + metadata.is_file() && !metadata.file_type().is_symlink(), + "canonical Agent Spec is not a real regular file: {}", + path.display() + ); + let declared = crate::discovery::parse_declared(&path)?; + Ok(declared.len() == 1 + && declared[0].host.as_deref() == Some(&key.host) + && declared[0].identity.as_deref() == Some(key.identity.as_str())) +} + +fn marker_state_exists(agent_dir: &Path) -> anyhow::Result { + let mut found = false; + for name in ["resources", "archive", "inbox"] { + let path = agent_dir.join(name); + match fs::symlink_metadata(&path) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => found = true, + Ok(_) => anyhow::bail!( + "agent state path is not a real directory: {}", + path.display() + ), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + } + match fs::symlink_metadata(agent_dir.join("status")) { + Ok(metadata) if metadata.is_file() && !metadata.file_type().is_symlink() => found = true, + Ok(_) => anyhow::bail!("agent status is not a real regular file"), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + let resources = agent_dir.join("resources"); + if resources.is_dir() { + for relative in ["inbox", "archive", "context", "context/decisions", "links"] { + match fs::symlink_metadata(resources.join(relative)) { + Ok(metadata) if metadata.is_dir() && !metadata.file_type().is_symlink() => {} + Ok(_) => anyhow::bail!("agent resource path is not a real directory"), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => return Err(error.into()), + } + } + } + Ok(found) +} + +fn apply_incomplete(root: &Path) -> bool { + fs::symlink_metadata(crate::catalog_lock::apply_marker_path(root)).is_ok() +} + +fn safe_component(value: &str) -> bool { + !value.is_empty() + && !matches!(value, "." | ".." | ".git" | ".st2") + && Path::new(value).components().count() == 1 } /// The default subject for a reply to a message whose subject was `original`: the original prefixed @@ -459,22 +754,32 @@ pub struct ThreadEntry { /// lives in BOTH agents' boxes, so scan every agent's inbox+archive). Returns the thread in reply-tree /// pre-order with depth; the root's `in-reply-to` chain is walked up first. Empty if `filename` isn't /// found. Cycles/dangling `in-reply-to` are handled defensively. -pub fn collect_thread(catalog_root: &Path, filename: &str) -> Vec { +pub fn collect_thread(catalog_root: &Path, filename: &str) -> anyhow::Result> { // Gather every message once (dedup by filename — the same file can appear in two boxes). - let found = crate::discover(catalog_root); let mut all: HashMap = HashMap::new(); - for spec in &found.specs { - let Some(dir) = spec.path.parent() else { - continue; - }; - for d in [inbox_dir(dir), archive_dir(dir)] { - for m in list_dir(&d).unwrap_or_default() { - all.entry(m.filename.clone()).or_insert(m); + let transition = crate::catalog_transaction::catalog_transition(catalog_root)?; + for agent in addressable_agent_dirs(catalog_root, "", transition.as_ref())? { + if let Some(capability) = agent.capability.as_ref() { + for components in [&["resources", "inbox"][..], &["resources", "archive"][..]] { + let Some(dir) = open_message_box(capability, components, false)? else { + continue; + }; + for message in list_dir(&crate::catalog_transaction::retained_dir_path(&dir)?) + .unwrap_or_default() + { + all.entry(message.filename.clone()).or_insert(message); + } + } + } else { + for dir in [inbox_dir(&agent.path), archive_dir(&agent.path)] { + for message in list_dir(&dir).unwrap_or_default() { + all.entry(message.filename.clone()).or_insert(message); + } } } } if !all.contains_key(filename) { - return Vec::new(); + return Ok(Vec::new()); } // Walk up `in-reply-to` to the thread root. @@ -524,7 +829,165 @@ pub fn collect_thread(catalog_root: &Path, filename: &str) -> Vec { } } } - out + Ok(out) +} + +pub fn send_to_resolved_inbox( + catalog_root: &Path, + recipient: &str, + this_host: &str, + from: &str, + subject: Option<&str>, + in_reply_to: Option<&str>, + tags: &[String], + body: &str, +) -> anyhow::Result { + let agent = match resolve_agent_handle(catalog_root, recipient, this_host)? { + Some(agent) => agent, + None => { + let discovered = crate::discover(catalog_root); + if crate::catalog_transaction::catalog_transition(catalog_root)?.is_none() + && !catalog_root.join(crate::catalog_lock::CONTROL_DIR).exists() + && discovered.specs.is_empty() + && discovered.errors.is_empty() + { + return send_to_inbox( + &catalog_root.join(recipient).join("inbox"), + from, + subject, + in_reply_to, + tags, + body, + ); + } + anyhow::bail!( + "no agent '{recipient}' found in catalog {}", + catalog_root.display() + ) + } + }; + test_capability_checkpoint(); + if let Some(capability) = agent.capability.as_ref() { + let inbox = open_message_box(capability, &["resources", "inbox"], true)? + .context("created inbox capability is missing")?; + send_to_inbox( + &crate::catalog_transaction::retained_dir_path(&inbox)?, + from, + subject, + in_reply_to, + tags, + body, + ) + } else { + send_to_inbox( + &inbox_dir(&agent.path), + from, + subject, + in_reply_to, + tags, + body, + ) + } +} + +#[cfg(debug_assertions)] +fn test_capability_checkpoint() { + let (Ok(ready), Ok(release)) = ( + std::env::var("ST2_TEST_MESSAGE_CAPABILITY_READY"), + std::env::var("ST2_TEST_MESSAGE_CAPABILITY_RELEASE"), + ) else { + return; + }; + let _ = fs::write(ready, b"ready"); + while !Path::new(&release).exists() { + std::thread::yield_now(); + } +} + +#[cfg(not(debug_assertions))] +fn test_capability_checkpoint() {} + +pub fn archive_resolved_message( + catalog_root: &Path, + identity: &str, + this_host: &str, + filename: &str, +) -> anyhow::Result<()> { + anyhow::ensure!( + is_message_filename(filename), + "invalid message filename {filename:?}" + ); + let agent = match resolve_agent_handle(catalog_root, identity, this_host)? { + Some(agent) => agent, + None => { + let discovered = crate::discover(catalog_root); + if crate::catalog_transaction::catalog_transition(catalog_root)?.is_none() + && !catalog_root.join(crate::catalog_lock::CONTROL_DIR).exists() + && discovered.specs.is_empty() + && discovered.errors.is_empty() + { + return archive_msg( + &catalog_root.join(identity).join("inbox"), + &catalog_root.join(identity).join("archive"), + filename, + ); + } + anyhow::bail!( + "no agent '{identity}' found in catalog {}", + catalog_root.display() + ) + } + }; + if let Some(capability) = agent.capability.as_ref() { + let inbox = open_message_box(capability, &["resources", "inbox"], false)? + .context("message inbox does not exist")?; + let archive = open_message_box(capability, &["resources", "archive"], true)? + .context("created archive capability is missing")?; + archive_msg( + &crate::catalog_transaction::retained_dir_path(&inbox)?, + &crate::catalog_transaction::retained_dir_path(&archive)?, + filename, + ) + } else { + archive_msg(&inbox_dir(&agent.path), &archive_dir(&agent.path), filename) + } +} + +fn open_message_box( + agent: &File, + components: &[&str], + create: bool, +) -> anyhow::Result> { + use std::ffi::CString; + use std::os::fd::AsRawFd as _; + let mut current = agent.try_clone()?; + for component in components { + match crate::catalog_transaction::openat_dir_nofollow( + ¤t, + std::ffi::OsStr::new(component), + ) { + Ok(next) => current = next, + Err(error) if error.kind() == std::io::ErrorKind::NotFound && !create => { + return Ok(None); + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => { + let name = CString::new(*component)?; + let result = unsafe { libc::mkdirat(current.as_raw_fd(), name.as_ptr(), 0o755) }; + if result != 0 { + let error = std::io::Error::last_os_error(); + if error.kind() != std::io::ErrorKind::AlreadyExists { + return Err(error.into()); + } + } + current = crate::catalog_transaction::openat_dir_nofollow( + ¤t, + std::ffi::OsStr::new(component), + )?; + } + Err(error) => return Err(error.into()), + } + } + Ok(Some(current)) } /// Archive `filename` from inbox to archive. @@ -533,6 +996,10 @@ pub fn collect_thread(catalog_root: &Path, filename: &str) -> Vec { /// it is the durable receipt and wins. Remove only a duplicate inbox copy, never overwrite the /// archived file. A source that disappeared concurrently is also success when the receipt exists. pub fn archive_msg(inbox_dir: &Path, archive_dir: &Path, filename: &str) -> anyhow::Result<()> { + anyhow::ensure!( + is_message_filename(filename), + "invalid message filename {filename:?}" + ); fs::create_dir_all(archive_dir)?; let source = inbox_dir.join(filename); let receipt = archive_dir.join(filename); @@ -737,7 +1204,14 @@ mod tests { #[test] fn external_inbox_rejects_unsafe_or_nested_identities() { let tmp = tempfile::tempdir().unwrap(); - for identity in ["", ".", "..", "nested/requester", "../requester", "/requester"] { + for identity in [ + "", + ".", + "..", + "nested/requester", + "../requester", + "/requester", + ] { assert!( ExternalInbox::new(tmp.path(), identity).is_err(), "accepted unsafe external identity {identity:?}" diff --git a/src/resource.rs b/src/resource.rs index 8592be28..c51ec8a0 100644 --- a/src/resource.rs +++ b/src/resource.rs @@ -35,14 +35,26 @@ fn valid_url(url: &str) -> bool { match url.split_once(':') { Some((scheme, _)) => { !scheme.is_empty() - && scheme.bytes().all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'+' || b == b'-' || b == b'.') + && scheme.bytes().all(|b| { + b.is_ascii_lowercase() + || b.is_ascii_digit() + || b == b'+' + || b == b'-' + || b == b'.' + }) } None => false, } } /// Render a resource record's file contents (frontmatter + body). -pub fn render(url: &str, title: Option<&str>, tags: &[String], relation: Option<&str>, body: &str) -> String { +pub fn render( + url: &str, + title: Option<&str>, + tags: &[String], + relation: Option<&str>, + body: &str, +) -> String { let mut s = String::from("---\n"); s.push_str(&format!("url: {url}\n")); if let Some(t) = title { @@ -71,7 +83,9 @@ fn parse(filename: &str, contents: &str) -> Resource { relation: None, body: String::new(), }; - let rest = contents.strip_prefix("---\n").or_else(|| contents.strip_prefix("---\r\n")); + let rest = contents + .strip_prefix("---\n") + .or_else(|| contents.strip_prefix("---\r\n")); if let Some(rest) = rest && let Some(end) = rest.find("\n---") { @@ -79,12 +93,20 @@ fn parse(filename: &str, contents: &str) -> Resource { let after = &rest[end + 1..]; let body = after.split_once('\n').map(|x| x.1).unwrap_or(""); for line in front.lines() { - let Some((k, v)) = line.split_once(':') else { continue }; + let Some((k, v)) = line.split_once(':') else { + continue; + }; let v = v.trim(); match k.trim() { "url" => r.url = v.to_string(), "title" => r.title = Some(v.to_string()), - "tags" => r.tags = v.split(',').map(|t| t.trim().to_string()).filter(|t| !t.is_empty()).collect(), + "tags" => { + r.tags = v + .split(',') + .map(|t| t.trim().to_string()) + .filter(|t| !t.is_empty()) + .collect() + } "relation" => r.relation = Some(v.to_string()), _ => {} } @@ -118,19 +140,27 @@ pub fn add( return Ok(filename); } } - anyhow::bail!("could not allocate a unique resource filename in {}", links_dir.display()) + anyhow::bail!( + "could not allocate a unique resource filename in {}", + links_dir.display() + ) } /// List an agent's resources, sorted by add time (filename). pub fn list(links_dir: &Path) -> Vec { let mut out = Vec::new(); - let Ok(rd) = fs::read_dir(links_dir) else { return out }; + let Ok(rd) = fs::read_dir(links_dir) else { + return out; + }; for e in rd.flatten() { let name = e.file_name().to_string_lossy().into_owned(); if !message::is_message_filename(&name) { continue; } - out.push(parse(&name, &fs::read_to_string(e.path()).unwrap_or_default())); + out.push(parse( + &name, + &fs::read_to_string(e.path()).unwrap_or_default(), + )); } out.sort_by(|a, b| a.filename.cmp(&b.filename)); out @@ -145,7 +175,12 @@ pub fn read(links_dir: &Path, filename: &str) -> anyhow::Result { /// Remove one resource record. pub fn remove(links_dir: &Path, filename: &str) -> anyhow::Result<()> { - fs::remove_file(links_dir.join(filename)).map_err(|e| anyhow::anyhow!("removing {filename}: {e}")) + anyhow::ensure!( + message::is_message_filename(filename), + "invalid resource filename {filename:?}" + ); + fs::remove_file(links_dir.join(filename)) + .map_err(|e| anyhow::anyhow!("removing {filename}: {e}")) } #[cfg(test)] @@ -157,7 +192,15 @@ mod tests { let tmp = tempfile::tempdir().unwrap(); let dir = links_dir(tmp.path()); let tags = vec!["pr".to_string(), "st2".to_string()]; - let f = add(&dir, "https://github.com/x/y/pull/1", Some("M2.6 PR"), &tags, Some("output"), "the resource PR").unwrap(); + let f = add( + &dir, + "https://github.com/x/y/pull/1", + Some("M2.6 PR"), + &tags, + Some("output"), + "the resource PR", + ) + .unwrap(); let all = list(&dir); assert_eq!(all.len(), 1); @@ -172,6 +215,19 @@ mod tests { assert!(list(&dir).is_empty()); } + #[test] + fn remove_rejects_noncanonical_leaf_paths() { + let tmp = tempfile::tempdir().unwrap(); + let links = links_dir(tmp.path()); + fs::create_dir_all(&links).unwrap(); + fs::write(tmp.path().join("outside.md"), "unchanged").unwrap(); + assert!(remove(&links, "../outside.md").is_err()); + assert_eq!( + fs::read_to_string(tmp.path().join("outside.md")).unwrap(), + "unchanged" + ); + } + #[test] fn rejects_non_urls() { let tmp = tempfile::tempdir().unwrap(); diff --git a/src/run.rs b/src/run.rs index 7ceeabdf..8e90dc79 100644 --- a/src/run.rs +++ b/src/run.rs @@ -1846,7 +1846,7 @@ pub fn surface_crash_loop(catalog_root: &Path, this_host: &str, cl: &CrashLoop) ); return; }; - let Some(agent_dir) = message::resolve_agent_dir(catalog_root, supervisor, this_host) else { + let Ok(Some(agent_dir)) = message::resolve_agent_dir(catalog_root, supervisor, this_host) else { eprintln!( "st2: crash-loop '{}': supervisor '{supervisor}' not found in the catalog to notify.", cl.pty_id diff --git a/src/validate.rs b/src/validate.rs index 7a543258..2fff34ac 100644 --- a/src/validate.rs +++ b/src/validate.rs @@ -263,9 +263,17 @@ fn validate_scoped(root: &Path, this_host: Option<&str>) -> Report { )); } - // Path fields must be absolute or $CATALOG-rooted, and must exist. + // Path fields must be absolute, $CATALOG-rooted, or the one canonical relative workspace. for (field, raw) in path_fields(s) { - if let Some(issue) = check_path(root, &rp, &ag, &field, &raw, runs_on_selected_host) { + if let Some(issue) = check_path( + root, + s.path.parent().unwrap_or(root), + &rp, + &ag, + &field, + &raw, + runs_on_selected_host, + ) { issues.push(issue); } } @@ -339,36 +347,31 @@ fn path_fields(s: &AgentSpec) -> Vec<(String, String)> { v } -/// Check one path field: `$CATALOG` expands to the catalog root; a path bearing any *other* `$VAR` is -/// skipped (an unset var is a literal token — do not guess). What remains must be absolute (R11: -/// final-spec paths are absolute or $CATALOG-rooted, never relative). Catalog-owned paths must -/// always exist; external paths are checked only for an agent assigned to the selected host. +/// Check one path field through the shared launch-equivalent resolver. Absolute paths remain valid; +/// a relative path must normalize to the declaring bundle's `.workspace`, and an unresolved +/// variable fails closed. Catalog-owned paths must always exist; external paths are checked only +/// for an agent assigned to the selected host. fn check_path( root: &Path, + spec_dir: &Path, rp: &str, ag: &Option, field: &str, raw: &str, check_external_presence: bool, ) -> Option { - let root_s = root.to_string_lossy(); - let expanded = raw - .replace("${CATALOG}", &root_s) - .replace("$CATALOG", &root_s); - if expanded.contains('$') { - return None; // another variable — cannot resolve without runtime env; do not guess - } - let p = Path::new(&expanded); - if !p.is_absolute() { - return Some(Issue::error( - "bad-path", - rp.to_string(), - ag.clone(), - format!( - "{field} '{raw}' is relative (final-spec paths must be absolute or $CATALOG-rooted)" - ), - )); - } + let resolved = match crate::expand::resolve_spec_path(raw, root, spec_dir) { + Ok(path) => path, + Err(error) => { + return Some(Issue::error( + "bad-path", + rp.to_string(), + ag.clone(), + format!("{field} '{raw}' is invalid: {error}"), + )); + } + }; + let p = &resolved; if !p.exists() { // A **catalog-rooted** path is the renderer's own output — its absence is a real render bug // (ERROR). An **external** absolute path is checked only for the selected run host; its diff --git a/tests/agent_presentation.rs b/tests/agent_presentation.rs index 7c50d4c0..09068398 100644 --- a/tests/agent_presentation.rs +++ b/tests/agent_presentation.rs @@ -3,10 +3,11 @@ use std::fs::OpenOptions; use std::os::fd::AsRawFd as _; use std::os::unix::fs::{MetadataExt as _, OpenOptionsExt as _, PermissionsExt as _}; use std::os::unix::process::CommandExt as _; +use std::os::unix::process::ExitStatusExt as _; use std::path::Path; use std::process::{Command, Stdio}; use std::thread; -use std::time::Duration; +use std::time::{Duration, Instant}; fn write(root: &Path, relative: &str, contents: &str) { let path = root.join(relative); @@ -35,6 +36,18 @@ fn run(root: &Path, command: &str, args: &[&str], actor: Option<&str>) -> std::p process.output().unwrap() } +fn wait_for_path(path: &Path) { + let deadline = Instant::now() + Duration::from_secs(2); + while !path.exists() { + assert!( + Instant::now() < deadline, + "timed out waiting for {}", + path.display() + ); + thread::sleep(Duration::from_millis(10)); + } +} + #[test] fn cli_sets_replaces_and_clears_fields_without_changing_identity_or_other_bytes() { let temporary = tempfile::tempdir().unwrap(); @@ -419,6 +432,7 @@ fn presentation_and_publication_contend_on_the_same_persistent_catalog_lock() { assert_eq!(unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_EX) }, 0); let inode = fs::metadata(&lock_path).unwrap().ino(); + let rename_attempt = temporary.path().join("rename-lock-attempt"); let mut rename = Command::new(env!("CARGO_BIN_EXE_st2")) .args([ "--catalog", @@ -430,13 +444,14 @@ fn presentation_and_publication_contend_on_the_same_persistent_catalog_lock() { "h", "--json", ]) + .env("ST2_TEST_CATALOG_LOCK_ATTEMPT", &rename_attempt) .env_remove("ST_AGENT") .stdin(Stdio::null()) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() .unwrap(); - thread::sleep(Duration::from_millis(50)); + wait_for_path(&rename_attempt); assert!(rename.try_wait().unwrap().is_none()); assert_eq!(unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_UN) }, 0); let output = rename.wait_with_output().unwrap(); @@ -448,7 +463,12 @@ fn presentation_and_publication_contend_on_the_same_persistent_catalog_lock() { let candidate = temporary.path().join("candidate.kdl"); fs::copy(&declaration_path, &candidate).unwrap(); + let digest = st2::agent_publish::digest_source(st2::agent_publish::PublishSource::Spec( + candidate.clone(), + )) + .unwrap(); assert_eq!(unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_EX) }, 0); + let publish_attempt = temporary.path().join("publish-lock-attempt"); let mut publish = Command::new(env!("CARGO_BIN_EXE_st2")) .args([ "agent", @@ -457,15 +477,18 @@ fn presentation_and_publication_contend_on_the_same_persistent_catalog_lock() { root.to_str().unwrap(), "--spec", candidate.to_str().unwrap(), + "--input-sha256", + &digest.sha256, "--expect-absent", "--json", ]) + .env("ST2_TEST_CATALOG_LOCK_ATTEMPT", &publish_attempt) .stdin(Stdio::null()) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() .unwrap(); - thread::sleep(Duration::from_millis(50)); + wait_for_path(&publish_attempt); assert!(publish.try_wait().unwrap().is_none()); assert_eq!(unsafe { libc::flock(lock.as_raw_fd(), libc::LOCK_UN) }, 0); let output = publish.wait_with_output().unwrap(); @@ -511,6 +534,170 @@ fn catalog_lock_refuses_a_symlinked_control_directory() { assert!(!outside.join("catalog-authoring.lock").exists()); } +#[test] +fn presentation_crash_stages_only_in_the_control_plane() { + let temporary = tempfile::tempdir().unwrap(); + let root = temporary.path().join("catalog"); + let original = declaration("worker", None, "catalog"); + write(&root, "agents/h/worker/agent.kdl", &original); + let crashed = Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "--catalog", + root.to_str().unwrap(), + "rename", + "h.worker", + "Build owner", + "--host", + "h", + ]) + .env_remove("ST_AGENT") + .env("ST2_TEST_AGENT_AUTHOR_CRASH_AFTER_TEMP", "1") + .output() + .unwrap(); + assert!(!crashed.status.success()); + assert_eq!( + crashed.status.signal(), + Some(libc::SIGABRT), + "status {:?}, stderr {}", + crashed.status, + String::from_utf8_lossy(&crashed.stderr) + ); + assert_eq!( + fs::read_to_string(root.join("agents/h/worker/agent.kdl")).unwrap(), + original + ); + assert!(!root.join(".st2/catalog-generation").exists()); + assert!( + fs::read_dir(root.join("agents/h/worker")) + .unwrap() + .all(|entry| !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with("agent-presentation-")) + ); + let retry = run( + &root, + "rename", + &["h.worker", "Build owner", "--host", "h", "--json"], + None, + ); + assert!( + retry.status.success(), + "{}", + String::from_utf8_lossy(&retry.stderr) + ); + assert_eq!( + fs::read_to_string(root.join(".st2/catalog-generation")).unwrap(), + "1\n" + ); +} + +#[test] +fn presentation_post_commit_generation_failure_is_fenced_and_recovered() { + let temporary = tempfile::tempdir().unwrap(); + let root = temporary.path().join("catalog"); + write( + &root, + "agents/h/worker/agent.kdl", + &declaration("worker", None, "catalog"), + ); + let failed = Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "--catalog", + root.to_str().unwrap(), + "rename", + "h.worker", + "Build owner", + "--host", + "h", + ]) + .env_remove("ST_AGENT") + .env("ST2_TEST_GENERATION_FAIL_AFTER_COMMIT", "1") + .output() + .unwrap(); + assert!(!failed.status.success()); + assert!( + fs::read_to_string(root.join("agents/h/worker/agent.kdl")) + .unwrap() + .contains("name \"Build owner\"") + ); + assert!(root.join(".st2/catalog-generation-incomplete").is_file()); + assert!(!root.join(".st2/catalog-generation").exists()); + let shared = Command::new(env!("CARGO_BIN_EXE_st2")) + .args(["agents", "--catalog", root.to_str().unwrap(), "--json"]) + .output() + .unwrap(); + assert!(!shared.status.success()); + + let recovered = run( + &root, + "rename", + &["h.worker", "Build owner", "--host", "h", "--json"], + None, + ); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert_eq!( + fs::read_to_string(root.join(".st2/catalog-generation")).unwrap(), + "1\n" + ); + assert!(!root.join(".st2/catalog-generation-incomplete").exists()); +} + +#[test] +fn control_directory_swap_cannot_redirect_presentation_staging() { + let temporary = tempfile::tempdir().unwrap(); + let root = temporary.path().join("catalog"); + write( + &root, + "agents/h/worker/agent.kdl", + &declaration("worker", None, "catalog"), + ); + let ready = temporary.path().join("ready"); + let release = temporary.path().join("release"); + let writer = Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "--catalog", + root.to_str().unwrap(), + "rename", + "h.worker", + "Build owner", + "--host", + "h", + ]) + .env_remove("ST_AGENT") + .env("ST2_TEST_CATALOG_LOCK_HELD_READY", &ready) + .env("ST2_TEST_CATALOG_LOCK_HELD_RELEASE", &release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for_path(&ready); + let retained = temporary.path().join("retained-control"); + fs::rename(root.join(".st2"), &retained).unwrap(); + let outside = temporary.path().join("outside-control"); + fs::create_dir(&outside).unwrap(); + std::os::unix::fs::symlink(&outside, root.join(".st2")).unwrap(); + fs::write(&release, "").unwrap(); + let writer = writer.wait_with_output().unwrap(); + assert!( + writer.status.success(), + "{}", + String::from_utf8_lossy(&writer.stderr) + ); + assert!( + fs::read_to_string(root.join("agents/h/worker/agent.kdl")) + .unwrap() + .contains("name \"Build owner\"") + ); + assert!(outside.read_dir().unwrap().next().is_none()); + assert!(retained.join("catalog-generation").is_file()); +} + #[test] fn catalog_lock_refuses_a_symlinked_lock_file() { use std::os::unix::fs::symlink; diff --git a/tests/agent_publish.rs b/tests/agent_publish.rs index e736eb12..f9469cef 100644 --- a/tests/agent_publish.rs +++ b/tests/agent_publish.rs @@ -1,5 +1,6 @@ use std::fs; use std::io::Write as _; +use std::os::unix::process::ExitStatusExt as _; use std::path::{Path, PathBuf}; use std::process::Stdio; use std::process::{Command, Output}; @@ -18,6 +19,7 @@ fn valid_spec(retired: bool) -> String { } fn publish(catalog: &Path, spec: &Path, expectation: &[&str]) -> Output { + let input_sha256 = sha256(&fs::read(spec).unwrap()); st2() .args([ "agent", @@ -27,12 +29,31 @@ fn publish(catalog: &Path, spec: &Path, expectation: &[&str]) -> Output { "--spec", spec.to_str().unwrap(), ]) + .args(["--input-sha256", &input_sha256]) .args(expectation) .arg("--json") .output() .unwrap() } +fn source_digest(flag: &str, path: &Path) -> String { + let output = st2() + .args(["agent", "digest", flag]) + .arg(path) + .arg("--json") + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + serde_json::from_slice::(&output.stdout).unwrap()["sha256"] + .as_str() + .unwrap() + .to_string() +} + fn sha256(bytes: &[u8]) -> String { format!("{:x}", Sha256::digest(bytes)) } @@ -59,6 +80,7 @@ fn spec_create_is_typed_and_idempotent() { assert_eq!(first["schema"], "st2.agent-publish.v1"); assert_eq!(first["status"], "published"); assert_eq!(first["busId"], "host.worker"); + assert_eq!(first["inputSha256"], sha256(valid_spec(false).as_bytes())); assert_eq!(first["afterSha256"], sha256(valid_spec(false).as_bytes())); assert_eq!( fs::read_to_string(target(&catalog)).unwrap(), @@ -75,6 +97,110 @@ fn spec_create_is_typed_and_idempotent() { assert_eq!(second["status"], "unchanged"); } +#[test] +fn caller_source_digest_rejects_mutation_and_symlink_swaps_before_publication() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + fs::create_dir(&catalog).unwrap(); + let spec = temp.path().join("candidate.kdl"); + fs::write(&spec, valid_spec(false)).unwrap(); + let reserved = source_digest("--spec", &spec); + + fs::write(&spec, valid_spec(true)).unwrap(); + let changed = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + spec.to_str().unwrap(), + "--input-sha256", + &reserved, + "--expect-absent", + ]) + .output() + .unwrap(); + assert!(!changed.status.success()); + assert!( + String::from_utf8_lossy(&changed.stderr).contains("input precondition failed"), + "{}", + String::from_utf8_lossy(&changed.stderr) + ); + assert!(!target(&catalog).exists()); + + let replacement = temp.path().join("replacement.kdl"); + fs::write(&replacement, valid_spec(false)).unwrap(); + fs::remove_file(&spec).unwrap(); + std::os::unix::fs::symlink(&replacement, &spec).unwrap(); + let swapped = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + spec.to_str().unwrap(), + "--input-sha256", + &reserved, + "--expect-absent", + ]) + .output() + .unwrap(); + assert!(!swapped.status.success()); + assert!(!target(&catalog).exists()); + + fs::remove_file(&spec).unwrap(); + let bundle = temp.path().join("bundle"); + fs::create_dir(&bundle).unwrap(); + fs::write(bundle.join("agent.kdl"), valid_spec(false)).unwrap(); + fs::write(bundle.join("asset.txt"), "reserved").unwrap(); + let bundle_reserved = source_digest("--bundle", &bundle); + fs::write(bundle.join("asset.txt"), "changed").unwrap(); + let changed_bundle = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--bundle", + bundle.to_str().unwrap(), + "--input-sha256", + &bundle_reserved, + "--expect-absent", + ]) + .output() + .unwrap(); + assert!(!changed_bundle.status.success()); + assert!( + String::from_utf8_lossy(&changed_bundle.stderr).contains("input precondition failed"), + "{}", + String::from_utf8_lossy(&changed_bundle.stderr) + ); + assert!(!target(&catalog).exists()); + + let external_asset = temp.path().join("external-asset"); + fs::write(&external_asset, "reserved").unwrap(); + fs::remove_file(bundle.join("asset.txt")).unwrap(); + std::os::unix::fs::symlink(&external_asset, bundle.join("asset.txt")).unwrap(); + let symlink_bundle = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--bundle", + bundle.to_str().unwrap(), + "--input-sha256", + &bundle_reserved, + "--expect-absent", + ]) + .output() + .unwrap(); + assert!(!symlink_bundle.status.success()); + assert!(!target(&catalog).exists()); +} + #[test] fn cas_rejects_stale_writers_and_preserves_resources() { let temp = tempfile::tempdir().unwrap(); @@ -264,6 +390,15 @@ fn incomplete_apply_marker_blocks_declarations_but_not_the_state_plane() { "incomplete apply reached runtime observation/action" ); + fs::write( + st2::catalog_lock::apply_marker_path(&catalog), + format!( + "{{\"schema\":\"st2.catalog-apply-incomplete.v1\",\"stageName\":\"catalog-apply-stage-{hash}\",\"expectedRootSha256\":\"{hash}\",\"preparedRootSha256\":\"{hash}\",\"originalPaths\":[\"agents/host/worker/agent.kdl\"]}}\n", + hash = "0".repeat(64) + ), + ) + .unwrap(); + let status = st2() .args([ "status", @@ -422,6 +557,7 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { fs::create_dir_all(bundle.join("resources/inbox")).unwrap(); fs::write(bundle.join("agent.kdl"), valid_spec(false)).unwrap(); fs::write(bundle.join("resources/inbox/kickoff.md"), "start").unwrap(); + let bundle_digest = source_digest("--bundle", &bundle); let first = st2() .args([ @@ -431,6 +567,8 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { catalog.to_str().unwrap(), "--bundle", bundle.to_str().unwrap(), + "--input-sha256", + &bundle_digest, "--expect-absent", "--json", ]) @@ -454,6 +592,8 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { catalog.to_str().unwrap(), "--bundle", bundle.to_str().unwrap(), + "--input-sha256", + &bundle_digest, "--expect-absent", "--json", ]) @@ -476,6 +616,8 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { catalog.to_str().unwrap(), "--bundle", bundle.to_str().unwrap(), + "--input-sha256", + &bundle_digest, "--expect-absent", ]) .output() @@ -485,6 +627,7 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { fs::write(&target_payload, "start").unwrap(); fs::write(bundle.join("resources/inbox/kickoff.md"), "different").unwrap(); + let changed_digest = source_digest("--bundle", &bundle); let mismatch = st2() .args([ "agent", @@ -493,6 +636,8 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { catalog.to_str().unwrap(), "--bundle", bundle.to_str().unwrap(), + "--input-sha256", + &changed_digest, "--expect-absent", ]) .output() @@ -500,6 +645,250 @@ fn bundle_is_atomic_create_only_and_retry_checks_the_full_payload() { assert!(!mismatch.status.success()); } +#[test] +fn spec_publish_crash_stages_only_in_the_control_plane() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let agent = catalog.join("agents/host/worker"); + fs::create_dir_all(&agent).unwrap(); + let original = valid_spec(false); + fs::write(agent.join("agent.kdl"), &original).unwrap(); + let candidate = temp.path().join("candidate.kdl"); + fs::write(&candidate, valid_spec(true)).unwrap(); + let digest = source_digest("--spec", &candidate); + let crashed = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(original.as_bytes()), + ]) + .env("ST2_TEST_AGENT_PUBLISH_CRASH_AFTER_TEMP", "1") + .output() + .unwrap(); + assert!(!crashed.status.success()); + assert_eq!(crashed.status.signal(), Some(libc::SIGABRT)); + assert_eq!( + fs::read_to_string(agent.join("agent.kdl")).unwrap(), + original + ); + assert!(!catalog.join(".st2/catalog-generation").exists()); + assert!(catalog.join(".st2/catalog-generation-incomplete").is_file()); + assert!(fs::read_dir(&agent).unwrap().all(|entry| { + !entry + .unwrap() + .file_name() + .to_string_lossy() + .starts_with("agent-publish-leaf-") + })); + let retry = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(original.as_bytes()), + ]) + .output() + .unwrap(); + assert!( + retry.status.success(), + "{}", + String::from_utf8_lossy(&retry.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join(".st2/catalog-generation")).unwrap(), + "2\n" + ); +} + +#[test] +fn publish_post_commit_generation_failure_is_fenced_and_recovered() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let agent = catalog.join("agents/host/worker"); + fs::create_dir_all(&agent).unwrap(); + let original = valid_spec(false); + let desired = valid_spec(true); + fs::write(agent.join("agent.kdl"), &original).unwrap(); + let candidate = temp.path().join("candidate.kdl"); + fs::write(&candidate, &desired).unwrap(); + let digest = source_digest("--spec", &candidate); + let failed = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(original.as_bytes()), + ]) + .env("ST2_TEST_GENERATION_FAIL_AFTER_COMMIT", "1") + .output() + .unwrap(); + assert!(!failed.status.success()); + assert_eq!( + fs::read_to_string(agent.join("agent.kdl")).unwrap(), + desired + ); + assert!(catalog.join(".st2/catalog-generation-incomplete").is_file()); + assert!(!catalog.join(".st2/catalog-generation").exists()); + let shared = st2() + .args(["agents", "--catalog", catalog.to_str().unwrap(), "--json"]) + .output() + .unwrap(); + assert!(!shared.status.success()); + + let recovered = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(desired.as_bytes()), + ]) + .output() + .unwrap(); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join(".st2/catalog-generation")).unwrap(), + "1\n" + ); + assert!(!catalog.join(".st2/catalog-generation-incomplete").exists()); +} + +#[test] +fn control_directory_swap_cannot_redirect_publication_staging() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let agent = catalog.join("agents/host/worker"); + fs::create_dir_all(&agent).unwrap(); + let original = valid_spec(false); + let desired = valid_spec(true); + fs::write(agent.join("agent.kdl"), &original).unwrap(); + let candidate = temp.path().join("candidate.kdl"); + fs::write(&candidate, &desired).unwrap(); + let digest = source_digest("--spec", &candidate); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let publisher = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(original.as_bytes()), + ]) + .env("ST2_TEST_CATALOG_LOCK_HELD_READY", &ready) + .env("ST2_TEST_CATALOG_LOCK_HELD_RELEASE", &release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for_path(&ready); + let retained = temp.path().join("retained-control"); + fs::rename(catalog.join(".st2"), &retained).unwrap(); + let outside = temp.path().join("outside-control"); + fs::create_dir(&outside).unwrap(); + std::os::unix::fs::symlink(&outside, catalog.join(".st2")).unwrap(); + fs::write(&release, "").unwrap(); + let publisher = publisher.wait_with_output().unwrap(); + assert!( + publisher.status.success(), + "{}", + String::from_utf8_lossy(&publisher.stderr) + ); + assert_eq!( + fs::read_to_string(agent.join("agent.kdl")).unwrap(), + desired + ); + assert!(outside.read_dir().unwrap().next().is_none()); + assert!(retained.join("catalog-generation").is_file()); +} + +#[test] +fn intermediate_host_swap_cannot_redirect_publication_outside_the_catalog() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let agent = catalog.join("agents/host/worker"); + fs::create_dir_all(&agent).unwrap(); + let original = valid_spec(false); + fs::write(agent.join("agent.kdl"), &original).unwrap(); + let candidate = temp.path().join("candidate.kdl"); + fs::write(&candidate, valid_spec(true)).unwrap(); + let digest = source_digest("--spec", &candidate); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let publisher = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + candidate.to_str().unwrap(), + "--input-sha256", + &digest, + "--expect-sha256", + &sha256(original.as_bytes()), + ]) + .env("ST2_TEST_AGENT_PUBLISH_READY", &ready) + .env("ST2_TEST_AGENT_PUBLISH_RELEASE", &release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for_path(&ready); + + let retained_host = temp.path().join("retained-host"); + fs::rename(catalog.join("agents/host"), &retained_host).unwrap(); + let outside = temp.path().join("outside-host"); + fs::create_dir_all(outside.join("worker")).unwrap(); + std::os::unix::fs::symlink(&outside, catalog.join("agents/host")).unwrap(); + fs::write(&release, "").unwrap(); + let publisher = publisher.wait_with_output().unwrap(); + assert!( + !publisher.status.success(), + "publication unexpectedly succeeded: {}", + String::from_utf8_lossy(&publisher.stdout) + ); + assert!(!outside.join("worker/agent.kdl").exists()); + assert_eq!( + fs::read_to_string(retained_host.join("worker/agent.kdl")).unwrap(), + original + ); + assert!(!catalog.join(".st2/catalog-generation").exists()); +} + #[test] fn compile_agent_is_not_a_cli_writer_anymore() { let output = st2().args(["compile-agent", "--help"]).output().unwrap(); @@ -517,6 +906,9 @@ fn compile_agent_is_not_a_cli_writer_anymore() { "--bundle", "--expect-absent", "--expect-sha256", + "--input-sha256", + "--prepared", + "--resume", "--json", ] { assert!( @@ -524,6 +916,12 @@ fn compile_agent_is_not_a_cli_writer_anymore() { "generated completion omitted {flag}" ); } + for command in ["digest", "catalog", "snapshot", "apply"] { + assert!( + completions.contains(command), + "generated completion omitted {command}" + ); + } assert!(!completions.contains("compile-agent")); } @@ -552,6 +950,7 @@ fn concurrent_publishers_serialize_and_only_one_wins_the_cas() { // Hold EX while both real CLI processes start, so neither can finish before the other is // contending on the same persistent lock. let gate = st2::CatalogLock::exclusive(&catalog).unwrap(); + let first_attempt = temp.path().join("first-publisher-lock-attempt"); let mut first = st2() .args([ "agent", @@ -562,12 +961,16 @@ fn concurrent_publishers_serialize_and_only_one_wins_the_cas() { one.to_str().unwrap(), "--expect-sha256", &expected, + "--input-sha256", + &sha256(fs::read(&one).unwrap().as_slice()), "--json", ]) + .env("ST2_TEST_CATALOG_LOCK_ATTEMPT", &first_attempt) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() .unwrap(); + let second_attempt = temp.path().join("second-publisher-lock-attempt"); let mut second = st2() .args([ "agent", @@ -578,13 +981,17 @@ fn concurrent_publishers_serialize_and_only_one_wins_the_cas() { two.to_str().unwrap(), "--expect-sha256", &expected, + "--input-sha256", + &sha256(fs::read(&two).unwrap().as_slice()), "--json", ]) + .env("ST2_TEST_CATALOG_LOCK_ATTEMPT", &second_attempt) .stdout(Stdio::piped()) .stderr(Stdio::piped()) .spawn() .unwrap(); - thread::sleep(Duration::from_millis(50)); + wait_for_path(&first_attempt); + wait_for_path(&second_attempt); assert!(first.try_wait().unwrap().is_none()); assert!(second.try_wait().unwrap().is_none()); drop(gate); @@ -672,6 +1079,8 @@ fn retirement_cannot_commit_between_reconcile_discovery_and_launch() { retired.to_str().unwrap(), "--expect-sha256", &sha256(active.as_bytes()), + "--input-sha256", + &sha256(fs::read(&retired).unwrap().as_slice()), "--json", ]) .stdout(Stdio::piped()) diff --git a/tests/catalog_apply.rs b/tests/catalog_apply.rs new file mode 100644 index 00000000..8f8f17f6 --- /dev/null +++ b/tests/catalog_apply.rs @@ -0,0 +1,2148 @@ +use std::ffi::CString; +use std::fs; +use std::io::Write as _; +use std::os::unix::ffi::OsStrExt as _; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Output, Stdio}; +use std::time::{Duration, Instant}; + +use serde_json::Value; +use sha2::{Digest as _, Sha256}; + +fn st2() -> Command { + Command::new(env!("CARGO_BIN_EXE_st2")) +} + +fn agent(identity: &str, retired: bool) -> String { + format!("agent \"{identity}\" {{\n host \"host\"\n retired #{retired}\n argv \"true\"\n}}\n") +} + +fn agent_dir(catalog: &Path, identity: &str) -> PathBuf { + catalog.join("agents/host").join(identity) +} + +fn write_agent(catalog: &Path, identity: &str, retired: bool) { + write_agent_for_host(catalog, "host", identity, retired); +} + +fn write_agent_for_host(catalog: &Path, host: &str, identity: &str, retired: bool) { + let dir = catalog.join("agents").join(host).join(identity); + fs::create_dir_all(&dir).unwrap(); + fs::write( + dir.join("agent.kdl"), + format!( + "agent \"{identity}\" {{\n host \"{host}\"\n retired #{retired}\n argv \"true\"\n}}\n" + ), + ) + .unwrap(); +} + +fn ensure_external_pty_config(catalog: &Path) { + let config = catalog.join("catalog.kdl"); + if !config.exists() { + fs::write( + config, + "catalog { pty-root \"/tmp/st2-catalog-transaction-test-pty\" }\n", + ) + .unwrap(); + } +} + +fn snapshot(catalog: &Path, output: &Path) -> Value { + ensure_external_pty_config(catalog); + let result = st2() + .args([ + "catalog", + "snapshot", + "--catalog", + catalog.to_str().unwrap(), + "--output", + output.to_str().unwrap(), + "--json", + ]) + .output() + .unwrap(); + assert!( + result.status.success(), + "snapshot stderr: {}", + String::from_utf8_lossy(&result.stderr) + ); + serde_json::from_slice(&result.stdout).unwrap() +} + +fn apply(catalog: &Path, prepared: &Path, expected: &str) -> Output { + st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + expected, + "--json", + ]) + .output() + .unwrap() +} + +fn resume(catalog: &Path) -> Output { + st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--resume", + "--json", + ]) + .output() + .unwrap() +} + +#[test] +fn catalog_apply_cli_exposes_exactly_the_two_closed_modes() { + let help = st2().args(["catalog", "apply", "--help"]).output().unwrap(); + assert!(help.status.success()); + let help = String::from_utf8(help.stdout).unwrap(); + for flag in ["--prepared", "--expect-sha256", "--resume", "--json"] { + assert!(help.contains(flag), "catalog apply help omitted {flag}"); + } + assert!(!help.contains("--expect-absent")); + + for args in [ + vec!["catalog", "apply"], + vec!["catalog", "apply", "--prepared", "/tmp/prepared"], + vec![ + "catalog", + "apply", + "--resume", + "--prepared", + "/tmp/prepared", + ], + vec![ + "catalog", + "apply", + "--resume", + "--expect-sha256", + "0000000000000000000000000000000000000000000000000000000000000000", + ], + ] { + let rejected = st2().args(args).output().unwrap(); + assert!( + !rejected.status.success(), + "catalog apply accepted an incomplete or ambiguous mode" + ); + } +} + +fn wait_for(path: &Path) { + let deadline = Instant::now() + Duration::from_secs(5); + while !path.exists() { + assert!( + Instant::now() < deadline, + "timed out waiting for {}", + path.display() + ); + std::thread::sleep(Duration::from_millis(2)); + } +} + +fn assert_no_writer_temporaries(root: &Path) { + if !root.exists() { + return; + } + for entry in fs::read_dir(root).unwrap() { + let entry = entry.unwrap(); + let path = entry.path(); + let name = entry.file_name(); + let name = name.to_string_lossy(); + assert!( + !name.starts_with(".agent.kdl.presentation-") + && !name.starts_with(".agent.kdl.publish-") + && !name.starts_with(".catalog-apply-file-"), + "writer temporary escaped into the declaration plane: {}", + path.display() + ); + if path.is_dir() && name != ".st2" { + assert_no_writer_temporaries(&path); + } + } +} + +fn write_test_marker(catalog: &Path, original_paths: &[&str]) { + fs::create_dir_all(catalog.join(".st2")).unwrap(); + let hash = "0".repeat(64); + let mut original_paths = original_paths.to_vec(); + original_paths.sort(); + let marker = serde_json::json!({ + "schema": "st2.catalog-apply-incomplete.v1", + "stageName": format!("catalog-apply-stage-{hash}"), + "expectedRootSha256": hash, + "preparedRootSha256": "0".repeat(64), + "originalPaths": original_paths, + }); + fs::write( + catalog.join(".st2/catalog-apply-incomplete"), + serde_json::to_vec(&marker).unwrap(), + ) + .unwrap(); +} + +fn send(catalog: &Path, recipient: &str, body: &str) -> Output { + let mut child = st2() + .args([ + "message", + "send", + recipient, + "--catalog", + catalog.to_str().unwrap(), + "--as", + "host.sender", + "--host", + "host", + ]) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + child + .stdin + .take() + .unwrap() + .write_all(body.as_bytes()) + .unwrap(); + child.wait_with_output().unwrap() +} + +fn run_with_stdin(args: &[&str], body: &str) -> Output { + let mut child = st2() + .args(args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + child + .stdin + .take() + .unwrap() + .write_all(body.as_bytes()) + .unwrap(); + child.wait_with_output().unwrap() +} + +fn paused_apply( + catalog: &Path, + prepared: &Path, + expected: &str, + point: &str, + ready: &Path, + release: &Path, +) -> Child { + let mut command = st2(); + command + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + expected, + "--json", + ]) + .env("ST2_TEST_CATALOG_APPLY_PAUSE_AT", point) + .env("ST2_TEST_CATALOG_APPLY_READY", ready) + .env("ST2_TEST_CATALOG_APPLY_RELEASE", release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + command.spawn().unwrap() +} + +#[test] +fn snapshot_is_typed_deterministic_and_excludes_state_and_workspaces() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let dir = agent_dir(&catalog, "worker"); + let workspace = temp.path().join("external-workspace"); + fs::create_dir(&workspace).unwrap(); + fs::create_dir_all(catalog.join("_templates")).unwrap(); + fs::write(catalog.join("_templates/prompt.md"), "prompt").unwrap(); + fs::write( + dir.join("agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n render {{ copy \"_templates/prompt.md\" \"prompt.md\" }}\n}}\n", + workspace.display() + ), + ) + .unwrap(); + fs::create_dir_all(dir.join("resources/inbox")).unwrap(); + fs::write(dir.join("resources/inbox/live.md"), "state").unwrap(); + fs::write(dir.join("status"), "busy").unwrap(); + fs::create_dir_all(dir.join("assets")).unwrap(); + fs::write(dir.join("assets/tool.sh"), "#!/bin/sh\ntrue\n").unwrap(); + fs::create_dir_all(catalog.join("workspaces/repo")).unwrap(); + fs::write(catalog.join("workspaces/repo/owned.txt"), "workspace").unwrap(); + + let output = temp.path().join("snapshot"); + let first = snapshot(&catalog, &output); + assert_eq!(first["schema"], "st2.catalog-snapshot.v1"); + assert_eq!(first["status"], "created"); + assert!(output.join("agents/host/worker/agent.kdl").is_file()); + assert!(output.join("agents/host/worker/assets/tool.sh").is_file()); + assert!(output.join("_templates/prompt.md").is_file()); + assert!(!output.join("agents/host/worker/resources").exists()); + assert!(!output.join("agents/host/worker/status").exists()); + assert!(!output.join("workspaces").exists()); + + let second = snapshot(&catalog, &output); + assert_eq!(second["status"], "unchanged"); + assert_eq!(second["rootSha256"], first["rootSha256"]); +} + +#[test] +fn complete_template_library_survives_unused_apply_and_supports_a_later_reference() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + fs::create_dir_all(catalog.join("_templates/nested")).unwrap(); + fs::write(catalog.join("_templates/used.md"), "used").unwrap(); + fs::write(catalog.join("_templates/future.md"), "future-v1").unwrap(); + fs::write(catalog.join("_templates/nested/alias.md"), "alias").unwrap(); + + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + assert_eq!( + fs::read_to_string(prepared.join("_templates/future.md")).unwrap(), + "future-v1" + ); + assert_eq!( + fs::read_to_string(prepared.join("_templates/nested/alias.md")).unwrap(), + "alias" + ); + + fs::write(prepared.join("_templates/future.md"), "future-v2").unwrap(); + fs::write(prepared.join("_templates/unreferenced.md"), "keep").unwrap(); + let applied = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join("_templates/unreferenced.md")).unwrap(), + "keep" + ); + + let workspace = temp.path().join("workspace"); + fs::create_dir(&workspace).unwrap(); + let next = temp.path().join("next"); + let current = snapshot(&catalog, &next); + fs::write( + next.join("agents/host/worker/agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n render {{ copy \"_templates/future.md\" \"future.md\" }}\n}}\n", + workspace.display() + ), + ) + .unwrap(); + let later = apply(&catalog, &next, current["rootSha256"].as_str().unwrap()); + assert!( + later.status.success(), + "{}", + String::from_utf8_lossy(&later.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join("_templates/future.md")).unwrap(), + "future-v2" + ); +} + +#[test] +fn template_library_rejects_malicious_nodes_and_every_explicit_bound() { + let temp = tempfile::tempdir().unwrap(); + for case in [ + "symlink", + "hardlink", + "fifo", + "depth", + "files", + "file-bytes", + "total-bytes", + ] { + let catalog = temp.path().join(format!("catalog-{case}")); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join(format!("prepared-{case}")); + let before = snapshot(&catalog, &prepared); + fs::create_dir(prepared.join("_templates")).unwrap(); + match case { + "symlink" => { + std::os::unix::fs::symlink(temp.path(), prepared.join("_templates/escape.md")) + .unwrap(); + } + "hardlink" => { + fs::write(prepared.join("_templates/original.md"), "same inode").unwrap(); + fs::hard_link( + prepared.join("_templates/original.md"), + prepared.join("_templates/alias.md"), + ) + .unwrap(); + } + "fifo" => { + let fifo = prepared.join("_templates/fifo"); + let fifo = CString::new(fifo.as_os_str().as_bytes()).unwrap(); + assert_eq!(unsafe { libc::mkfifo(fifo.as_ptr(), 0o600) }, 0); + } + "depth" => { + let mut dir = prepared.join("_templates"); + for index in 0..9 { + dir.push(format!("d{index}")); + fs::create_dir(&dir).unwrap(); + } + fs::write(dir.join("too-deep.md"), "deep").unwrap(); + } + "files" => { + for index in 0..257 { + fs::write(prepared.join(format!("_templates/{index}.md")), "").unwrap(); + } + } + "file-bytes" => { + let file = fs::File::create(prepared.join("_templates/large.md")).unwrap(); + file.set_len(1024 * 1024 + 1).unwrap(); + } + "total-bytes" => { + for index in 0..33 { + let file = + fs::File::create(prepared.join(format!("_templates/large-{index}.md"))) + .unwrap(); + file.set_len(1024 * 1024).unwrap(); + } + } + _ => unreachable!(), + } + let output = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!( + !output.status.success(), + "template case {case} unexpectedly succeeded" + ); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + } +} + +#[test] +fn apply_is_cas_guarded_idempotent_and_preserves_orphan_state() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let dir = agent_dir(&catalog, "worker"); + fs::create_dir_all(dir.join("resources/inbox")).unwrap(); + fs::write(dir.join("resources/inbox/live.md"), "keep").unwrap(); + fs::write(dir.join("status"), "busy").unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::remove_file(prepared.join("agents/host/worker/agent.kdl")).unwrap(); + fs::remove_dir_all(prepared.join("agents/host/worker")).unwrap(); + fs::remove_dir(prepared.join("agents/host")).unwrap(); + fs::remove_dir(prepared.join("agents")).unwrap(); + + let stale = apply(&catalog, &prepared, &"0".repeat(64)); + assert!(!stale.status.success()); + assert!(dir.join("agent.kdl").exists()); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + + let applied = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!( + applied.status.success(), + "apply stderr: {}", + String::from_utf8_lossy(&applied.stderr) + ); + let applied: Value = serde_json::from_slice(&applied.stdout).unwrap(); + assert_eq!(applied["schema"], "st2.catalog-apply.v1"); + assert_eq!(applied["status"], "applied"); + assert!(!dir.join("agent.kdl").exists()); + assert_eq!( + fs::read_to_string(dir.join("resources/inbox/live.md")).unwrap(), + "keep" + ); + assert_eq!(fs::read_to_string(dir.join("status")).unwrap(), "busy"); + + let after = snapshot(&catalog, &temp.path().join("after")); + assert_eq!(after["rootSha256"], applied["afterSha256"]); + let unchanged = apply(&catalog, &prepared, after["rootSha256"].as_str().unwrap()); + assert!(unchanged.status.success()); + let unchanged: Value = serde_json::from_slice(&unchanged.stdout).unwrap(); + assert_eq!(unchanged["status"], "unchanged"); +} + +#[test] +fn workspace_facts_are_empty_in_prepared_admitted_against_live_and_never_applied() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let workspace = catalog.join("agents/host/worker/.workspace"); + fs::create_dir_all(&workspace).unwrap(); + fs::write(workspace.join("live.txt"), "preserve").unwrap(); + fs::write( + catalog.join("agents/host/worker/agent.kdl"), + "agent \"worker\" {\n host \"host\"\n workspace \".workspace\"\n argv \"true\"\n}\n", + ) + .unwrap(); + let task_workspace = catalog.join("agents/host/tasker/.workspace"); + fs::create_dir_all(&task_workspace).unwrap(); + fs::write(task_workspace.join("task-live.txt"), "task preserve").unwrap(); + fs::write( + catalog.join("agents/host/tasker/agent.kdl"), + "agent \"tasker\" {\n host \"host\"\n pty \"work\" {\n cwd \".workspace\"\n argv \"true\"\n }\n}\n", + ) + .unwrap(); + + for case in ["missing", "content"] { + let invalid = temp.path().join(format!("invalid-{case}")); + let current = snapshot(&catalog, &invalid); + let fact = invalid.join("agents/host/worker/.workspace"); + if case == "missing" { + fs::remove_dir(&fact).unwrap(); + } else { + let secret = temp.path().join("workspace-secret"); + fs::write(&secret, "must never be opened or copied").unwrap(); + std::os::unix::fs::symlink(&secret, fact.join("forbidden-link")).unwrap(); + } + let rejected = apply(&catalog, &invalid, current["rootSha256"].as_str().unwrap()); + assert!( + !rejected.status.success(), + "prepared workspace {case} unexpectedly succeeded" + ); + if case == "content" { + assert!( + String::from_utf8_lossy(&rejected.stderr) + .contains("prepared workspace fact must be empty"), + "{}", + String::from_utf8_lossy(&rejected.stderr) + ); + } + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + assert_eq!( + fs::read_to_string(workspace.join("live.txt")).unwrap(), + "preserve" + ); + } + + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + let prepared_workspace = prepared.join("agents/host/worker/.workspace"); + assert!(prepared_workspace.is_dir()); + assert!(prepared_workspace.read_dir().unwrap().next().is_none()); + let prepared_task_workspace = prepared.join("agents/host/tasker/.workspace"); + assert!(prepared_task_workspace.is_dir()); + assert!(prepared_task_workspace.read_dir().unwrap().next().is_none()); + + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + "agent \"worker\" {\n host \"host\"\n role \"updated\"\n workspace \"./.workspace\"\n argv \"true\"\n}\n", + ) + .unwrap(); + let applied = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); + assert_eq!( + fs::read_to_string(workspace.join("live.txt")).unwrap(), + "preserve" + ); + assert_eq!( + fs::read_to_string(task_workspace.join("task-live.txt")).unwrap(), + "task preserve" + ); + let unchanged = apply( + &catalog, + &prepared, + serde_json::from_slice::(&applied.stdout).unwrap()["afterSha256"] + .as_str() + .unwrap(), + ); + assert!( + unchanged.status.success(), + "{}", + String::from_utf8_lossy(&unchanged.stderr) + ); + assert_eq!( + serde_json::from_slice::(&unchanged.stdout).unwrap()["status"], + "unchanged" + ); + + let next_workspace = catalog.join("agents/host/renamed/.workspace"); + let next = temp.path().join("next"); + let current = snapshot(&catalog, &next); + fs::create_dir_all(&next_workspace).unwrap(); + fs::write(next_workspace.join("next.txt"), "also preserve").unwrap(); + fs::remove_dir(next.join("agents/host/worker/.workspace")).unwrap(); + fs::remove_file(next.join("agents/host/worker/agent.kdl")).unwrap(); + fs::remove_dir(next.join("agents/host/worker")).unwrap(); + fs::create_dir_all(next.join("agents/host/renamed/.workspace")).unwrap(); + fs::write( + next.join("agents/host/renamed/agent.kdl"), + "agent \"renamed\" {\n host \"host\"\n workspace \".workspace\"\n argv \"true\"\n}\n", + ) + .unwrap(); + let changed = apply(&catalog, &next, current["rootSha256"].as_str().unwrap()); + assert!( + changed.status.success(), + "{}", + String::from_utf8_lossy(&changed.stderr) + ); + assert_eq!( + fs::read_to_string(workspace.join("live.txt")).unwrap(), + "preserve" + ); + assert_eq!( + fs::read_to_string(next_workspace.join("next.txt")).unwrap(), + "also preserve" + ); + let after = temp.path().join("after-workspace-move"); + snapshot(&catalog, &after); + assert!(!after.join("agents/host/worker/.workspace").exists()); + assert!( + after + .join("agents/host/renamed/.workspace") + .read_dir() + .unwrap() + .next() + .is_none() + ); +} + +#[test] +fn environment_expanded_relative_workspace_is_projected_and_admitted_consistently() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + let workspace = catalog.join("agents/host/worker/.workspace"); + fs::create_dir_all(&workspace).unwrap(); + fs::write(workspace.join("live.txt"), "preserve").unwrap(); + fs::write( + catalog.join("agents/host/worker/agent.kdl"), + "agent \"worker\" {\n host \"host\"\n workspace \".workspace\"\n argv \"true\"\n}\n", + ) + .unwrap(); + + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + "agent \"worker\" {\n host \"host\"\n role \"updated\"\n workspace \"$ST2_TEST_WORKSPACE\"\n argv \"true\"\n}\n", + ) + .unwrap(); + let applied = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_WORKSPACE", ".workspace") + .output() + .unwrap(); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); + assert_eq!( + fs::read_to_string(workspace.join("live.txt")).unwrap(), + "preserve" + ); +} + +#[test] +fn workspace_fact_symlink_ancestry_fails_before_publication() { + let temp = tempfile::tempdir().unwrap(); + let target = temp.path().join("target"); + let old_workspace = temp.path().join("old-workspace"); + fs::create_dir(&old_workspace).unwrap(); + fs::create_dir_all(target.join("agents/host/worker")).unwrap(); + fs::write( + target.join("agents/host/worker/agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n}}\n", + old_workspace.display() + ), + ) + .unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&target, &prepared); + + let external = temp.path().join("external"); + fs::create_dir(&external).unwrap(); + std::os::unix::fs::symlink(&external, target.join("agents/host/worker/.workspace")).unwrap(); + let workspace = target.join("agents/host/worker/.workspace"); + + fs::create_dir_all(prepared.join("agents/host/worker/.workspace")).unwrap(); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n}}\n", + workspace.display() + ), + ) + .unwrap(); + let output = apply(&target, &prepared, before["rootSha256"].as_str().unwrap()); + assert!(!output.status.success()); + assert!( + fs::read_to_string(target.join("agents/host/worker/agent.kdl")) + .unwrap() + .contains(old_workspace.to_str().unwrap()) + ); + assert!(!target.join(".st2/catalog-apply-incomplete").exists()); +} + +#[test] +fn prepared_state_symlinks_and_pty_root_changes_fail_before_a_marker() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let original = snapshot(&catalog, &temp.path().join("original")); + let expected = original["rootSha256"].as_str().unwrap(); + + for case in ["state", "workspace", "symlink", "fifo", "pty-root"] { + let prepared = temp.path().join(format!("prepared-{case}")); + snapshot(&catalog, &prepared); + match case { + "state" => { + fs::create_dir_all(prepared.join("agents/host/worker/resources/inbox")).unwrap(); + fs::write( + prepared.join("agents/host/worker/resources/inbox/message.md"), + "forbidden", + ) + .unwrap(); + } + "workspace" => { + fs::create_dir_all(prepared.join("workspaces/repo")).unwrap(); + fs::write(prepared.join("workspaces/repo/file"), "not declarations").unwrap(); + } + "symlink" => { + std::os::unix::fs::symlink("/tmp", prepared.join("agents/host/worker/assets-link")) + .unwrap(); + } + "fifo" => { + let fifo = prepared.join("agents/host/worker/fifo"); + let fifo = CString::new(fifo.as_os_str().as_bytes()).unwrap(); + assert_eq!(unsafe { libc::mkfifo(fifo.as_ptr(), 0o600) }, 0); + } + "pty-root" => { + fs::write( + prepared.join("catalog.kdl"), + "catalog { pty-root \"/tmp/other-pty\" }\n", + ) + .unwrap(); + } + _ => unreachable!(), + } + let result = apply(&catalog, &prepared, expected); + assert!( + !result.status.success(), + "case {case} unexpectedly succeeded" + ); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + } +} + +#[test] +fn apply_v1_requires_a_declared_pty_root_outside_the_catalog() { + let temp = tempfile::tempdir().unwrap(); + for case in ["default", "relative", "catalog-variable"] { + let catalog = temp.path().join(format!("catalog-{case}")); + write_agent(&catalog, "worker", false); + match case { + "default" => {} + "relative" => fs::write( + catalog.join("catalog.kdl"), + "catalog { pty-root \"registry\" }\n", + ) + .unwrap(), + "catalog-variable" => fs::write( + catalog.join("catalog.kdl"), + "catalog { pty-root \"$CATALOG/../catalog-catalog-variable/pty\" }\n", + ) + .unwrap(), + _ => unreachable!(), + } + let prepared = temp.path().join(format!("prepared-{case}")); + let captured = st2() + .args([ + "catalog", + "snapshot", + "--catalog", + catalog.to_str().unwrap(), + "--output", + prepared.to_str().unwrap(), + "--json", + ]) + .output() + .unwrap(); + assert!(captured.status.success()); + let captured: Value = serde_json::from_slice(&captured.stdout).unwrap(); + let rejected = apply( + &catalog, + &prepared, + captured["rootSha256"].as_str().unwrap(), + ); + assert!( + !rejected.status.success(), + "{case} unexpectedly admitted: {}", + String::from_utf8_lossy(&rejected.stderr) + ); + assert!( + String::from_utf8_lossy(&rejected.stderr).contains("requires pty-root outside") + || String::from_utf8_lossy(&rejected.stderr) + .contains("requires an explicit external pty-root") + ); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + } +} + +#[test] +fn retained_capture_refuses_a_destination_inside_its_source_before_traversal() { + let temp = tempfile::tempdir().unwrap(); + let bundle = temp.path().join("bundle"); + fs::create_dir(&bundle).unwrap(); + fs::write(bundle.join("agent.kdl"), agent("worker", false)).unwrap(); + let digest = st2() + .args(["agent", "digest", "--bundle"]) + .arg(&bundle) + .env("TMPDIR", &bundle) + .output() + .unwrap(); + assert!(!digest.status.success()); + assert!( + String::from_utf8_lossy(&digest.stderr).contains("is contained by source"), + "{}", + String::from_utf8_lossy(&digest.stderr) + ); + assert_eq!(bundle.read_dir().unwrap().count(), 1); + + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + let result = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("TMPDIR", &prepared) + .output() + .unwrap(); + assert!(!result.status.success()); + assert!( + String::from_utf8_lossy(&result.stderr).contains("is contained by source"), + "{}", + String::from_utf8_lossy(&result.stderr) + ); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); +} + +#[test] +fn concurrent_control_creation_fsyncs_both_the_creator_and_racing_observer_paths() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let release = temp.path().join("release"); + let mut children = Vec::new(); + for index in 0..2 { + let ready = temp.path().join(format!("ready-{index}")); + let branch = temp.path().join(format!("branch-{index}")); + let output = temp.path().join(format!("snapshot-{index}")); + let mut command = st2(); + command + .args([ + "catalog", + "snapshot", + "--catalog", + catalog.to_str().unwrap(), + "--output", + output.to_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_CATALOG_CONTROL_READY", &ready) + .env("ST2_TEST_CATALOG_CONTROL_RELEASE", &release) + .env("ST2_TEST_CATALOG_CONTROL_BRANCH", &branch) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + children.push((ready, branch, command.spawn().unwrap())); + } + wait_for(&children[0].0); + wait_for(&children[1].0); + fs::write(&release, "").unwrap(); + + let mut branches = Vec::new(); + for (_, branch, child) in children { + let output = child.wait_with_output().unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + branches.push(fs::read_to_string(branch).unwrap()); + } + branches.sort(); + assert_eq!(branches, ["created", "raced"]); + assert!(catalog.join(".st2/catalog-authoring.lock").is_file()); + + // A third contender can first observe the directory after mkdir but before the creator's + // parent fsync. The observer must independently fsync the catalog before using the control dir. + let observed_catalog = temp.path().join("observed-catalog"); + write_agent(&observed_catalog, "worker", false); + let created_ready = temp.path().join("created-ready"); + let created_release = temp.path().join("created-release"); + let creator_branch = temp.path().join("creator-branch"); + let creator = st2() + .args([ + "catalog", + "snapshot", + "--catalog", + observed_catalog.to_str().unwrap(), + "--output", + temp.path().join("creator-snapshot").to_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_CATALOG_CONTROL_CREATED_READY", &created_ready) + .env("ST2_TEST_CATALOG_CONTROL_CREATED_RELEASE", &created_release) + .env("ST2_TEST_CATALOG_CONTROL_BRANCH", &creator_branch) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&created_ready); + + let observer_branch = temp.path().join("observer-branch"); + let observer = st2() + .args([ + "catalog", + "snapshot", + "--catalog", + observed_catalog.to_str().unwrap(), + "--output", + temp.path().join("observer-snapshot").to_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_CATALOG_CONTROL_BRANCH", &observer_branch) + .output() + .unwrap(); + assert!( + observer.status.success(), + "{}", + String::from_utf8_lossy(&observer.stderr) + ); + assert_eq!(fs::read_to_string(observer_branch).unwrap(), "observed"); + + fs::write(&created_release, "").unwrap(); + let creator = creator.wait_with_output().unwrap(); + assert!( + creator.status.success(), + "{}", + String::from_utf8_lossy(&creator.stderr) + ); + assert_eq!(fs::read_to_string(creator_branch).unwrap(), "created"); +} + +#[test] +fn crashes_recover_from_the_durable_stage_without_rechecking_partial_live_state() { + for point in [ + "marker-created", + "leaf-staged", + "mid-write", + "before-verify", + "before-clear", + ] { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", true), + ) + .unwrap(); + if point == "leaf-staged" { + fs::write( + agent_dir(&catalog, "worker").join(".agent.kdl.presentation-orphan"), + "stale", + ) + .unwrap(); + fs::create_dir_all(catalog.join("_templates")).unwrap(); + fs::write( + catalog.join("_templates/.agent.kdl.publish-orphan"), + "stale", + ) + .unwrap(); + } + + let crashed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_CATALOG_APPLY_CRASH_AT", point) + .output() + .unwrap(); + assert!( + !crashed.status.success(), + "crash point {point} did not abort" + ); + assert!(catalog.join(".st2/catalog-apply-incomplete").is_file()); + + let competing = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!(!competing.status.success()); + assert!( + String::from_utf8_lossy(&competing.stderr) + .contains("recover only with `catalog apply --resume`") + ); + fs::remove_dir_all(&prepared).unwrap(); + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "recovery {point}: {}", + String::from_utf8_lossy(&recovered.stderr) + ); + let recovered: Value = serde_json::from_slice(&recovered.stdout).unwrap(); + assert_eq!(recovered["status"], "applied"); + assert_eq!(recovered["recovered"], true); + assert!(recovered["prepared"].is_null()); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + agent("worker", true) + ); + assert_no_writer_temporaries(&catalog); + let verified = snapshot(&catalog, &temp.path().join("verified")); + assert_eq!(verified["rootSha256"], recovered["afterSha256"]); + } +} + +#[test] +fn a_crash_before_new_identity_publication_resumes_from_control_plane_staging() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + write_agent(&prepared, "new", false); + + let crashed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("ST2_TEST_CATALOG_APPLY_CRASH_AT", "identity-staged") + .output() + .unwrap(); + assert!(!crashed.status.success()); + assert!(!agent_dir(&catalog, "new").exists()); + assert!(catalog.join(".st2/catalog-apply-incomplete").is_file()); + fs::remove_dir_all(&prepared).unwrap(); + + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert!(agent_dir(&catalog, "new").join("agent.kdl").is_file()); + assert_no_writer_temporaries(&catalog); +} + +#[test] +fn apply_post_commit_generation_failure_is_fenced_and_recovered() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", true), + ) + .unwrap(); + let failed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_GENERATION_FAIL_AFTER_COMMIT", "1") + .output() + .unwrap(); + assert!(!failed.status.success()); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + fs::read_to_string(prepared.join("agents/host/worker/agent.kdl")).unwrap() + ); + assert!(catalog.join(".st2/catalog-apply-incomplete").is_file()); + assert!(catalog.join(".st2/catalog-generation-incomplete").is_file()); + assert!(!catalog.join(".st2/catalog-generation").exists()); + let shared = st2() + .args(["agents", "--catalog", catalog.to_str().unwrap(), "--json"]) + .output() + .unwrap(); + assert!(!shared.status.success()); + + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join(".st2/catalog-generation")).unwrap(), + "2\n" + ); + assert!(!catalog.join(".st2/catalog-generation-incomplete").exists()); + assert!(!catalog.join(".st2/catalog-apply-incomplete").exists()); +} + +#[test] +fn control_directory_swap_cannot_redirect_apply_leaf_or_identity_staging() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/old/agent.kdl"), + agent("old", true), + ) + .unwrap(); + write_agent(&prepared, "new", false); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let apply = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + "--json", + ]) + .env("ST2_TEST_CATALOG_LOCK_HELD_READY", &ready) + .env("ST2_TEST_CATALOG_LOCK_HELD_RELEASE", &release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&ready); + let retained = temp.path().join("retained-control"); + fs::rename(catalog.join(".st2"), &retained).unwrap(); + let outside = temp.path().join("outside-control"); + fs::create_dir(&outside).unwrap(); + std::os::unix::fs::symlink(&outside, catalog.join(".st2")).unwrap(); + fs::write(&release, "").unwrap(); + let apply = apply.wait_with_output().unwrap(); + assert!( + apply.status.success(), + "{}", + String::from_utf8_lossy(&apply.stderr) + ); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "old").join("agent.kdl")).unwrap(), + fs::read_to_string(prepared.join("agents/host/old/agent.kdl")).unwrap() + ); + assert!(agent_dir(&catalog, "new").join("agent.kdl").is_file()); + assert!(outside.read_dir().unwrap().next().is_none()); + assert!(retained.join("catalog-generation").is_file()); + assert!(!retained.join("catalog-apply-incomplete").exists()); +} + +#[test] +fn cross_device_leaf_publication_fails_closed_and_remains_source_free_resumable() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", true), + ) + .unwrap(); + let failed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("ST2_TEST_CATALOG_APPLY_EXDEV_AT", "leaf-staged") + .output() + .unwrap(); + assert!(!failed.status.success()); + assert!( + String::from_utf8_lossy(&failed.stderr).contains("must share one filesystem"), + "{}", + String::from_utf8_lossy(&failed.stderr) + ); + assert!(catalog.join(".st2/catalog-apply-incomplete").is_file()); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + agent("worker", false) + ); + fs::remove_dir_all(&prepared).unwrap(); + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + agent("worker", true) + ); +} + +#[test] +fn cross_device_identity_publication_fails_closed_and_remains_source_free_resumable() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + write_agent(&prepared, "new", false); + let failed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("ST2_TEST_CATALOG_APPLY_EXDEV_AT", "identity-staged") + .output() + .unwrap(); + assert!(!failed.status.success()); + assert!(String::from_utf8_lossy(&failed.stderr).contains("must share one filesystem")); + assert!(catalog.join(".st2/catalog-apply-incomplete").is_file()); + assert!(!agent_dir(&catalog, "new").exists()); + fs::remove_dir_all(&prepared).unwrap(); + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert!(agent_dir(&catalog, "new").join("agent.kdl").is_file()); +} + +#[test] +fn recovery_does_not_need_a_partially_broken_old_render_graph() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let workspace = temp.path().join("workspace"); + fs::create_dir(&workspace).unwrap(); + fs::create_dir_all(catalog.join("_templates")).unwrap(); + fs::write(catalog.join("_templates/old.md"), "old").unwrap(); + fs::write( + agent_dir(&catalog, "worker").join("agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n render {{ copy \"_templates/old.md\" \"prompt.md\" }}\n}}\n", + workspace.display() + ), + ) + .unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write(prepared.join("_templates/new.md"), "new").unwrap(); + fs::remove_file(prepared.join("_templates/old.md")).unwrap(); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + format!( + "agent \"worker\" {{\n host \"host\"\n workspace \"{}\"\n argv \"true\"\n render {{ copy \"_templates/new.md\" \"prompt.md\" }}\n}}\n", + workspace.display() + ), + ) + .unwrap(); + + let crashed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("ST2_TEST_CATALOG_APPLY_CRASH_AT", "mid-delete") + .output() + .unwrap(); + assert!(!crashed.status.success()); + assert!(!catalog.join("_templates/old.md").exists()); + + fs::remove_dir_all(&prepared).unwrap(); + let recovered = resume(&catalog); + assert!( + recovered.status.success(), + "{}", + String::from_utf8_lossy(&recovered.stderr) + ); + assert_eq!( + fs::read_to_string(catalog.join("_templates/new.md")).unwrap(), + "new" + ); + assert!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")) + .unwrap() + .contains("_templates/new.md") + ); +} + +#[test] +fn mismatched_recovery_and_malformed_markers_remain_fenced_without_mutation() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", true), + ) + .unwrap(); + let crashed = st2() + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + before["rootSha256"].as_str().unwrap(), + ]) + .env("ST2_TEST_CATALOG_APPLY_CRASH_AT", "marker-created") + .output() + .unwrap(); + assert!(!crashed.status.success()); + let marker = catalog.join(".st2/catalog-apply-incomplete"); + assert!(marker.is_file()); + let marker_json: Value = serde_json::from_slice(&fs::read(&marker).unwrap()).unwrap(); + assert_eq!(marker_json["schema"], "st2.catalog-apply-incomplete.v1"); + assert_eq!(marker_json["expectedRootSha256"], before["rootSha256"]); + assert!( + marker_json["stageName"] + .as_str() + .unwrap() + .starts_with("catalog-apply-stage-") + ); + assert!(marker_json["preparedRootSha256"].as_str().is_some()); + assert!(marker_json["originalPaths"].is_array()); + + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", false), + ) + .unwrap(); + let mismatched = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!(!mismatched.status.success()); + assert!(marker.is_file()); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + agent("worker", false) + ); + + let live_workspace = agent_dir(&catalog, "worker").join(".workspace"); + fs::create_dir_all(&live_workspace).unwrap(); + let live_file = live_workspace.join("live.txt"); + fs::write(&live_file, "must survive forged recovery").unwrap(); + let mut forged = marker_json.clone(); + let original_paths = forged["originalPaths"].as_array_mut().unwrap(); + original_paths.push(Value::String( + "agents/host/worker/.workspace/live.txt".to_owned(), + )); + original_paths.sort_by(|left, right| left.as_str().cmp(&right.as_str())); + fs::write(&marker, serde_json::to_vec(&forged).unwrap()).unwrap(); + let forged_recovery = resume(&catalog); + assert!(!forged_recovery.status.success()); + assert!( + String::from_utf8_lossy(&forged_recovery.stderr).contains("workspace or state-plane path"), + "{}", + String::from_utf8_lossy(&forged_recovery.stderr) + ); + assert_eq!( + fs::read_to_string(&live_file).unwrap(), + "must survive forged recovery" + ); + assert!(marker.is_file()); + + fs::write(&marker, "{broken").unwrap(); + let malformed = resume(&catalog); + assert!(!malformed.status.success()); + assert_eq!(fs::read_to_string(&marker).unwrap(), "{broken"); +} + +#[test] +fn marker_time_state_routes_existing_orphans_but_never_flat_falls_back_for_new_agents() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let old = agent_dir(&catalog, "old"); + fs::create_dir_all(old.join("resources/inbox")).unwrap(); + write_agent(&catalog, "mix.sup", false); + fs::create_dir_all(agent_dir(&catalog, "mix.sup").join("resources/inbox")).unwrap(); + write_agent(&catalog, "remote.worker", false); + fs::create_dir_all(agent_dir(&catalog, "remote.worker").join("resources/inbox")).unwrap(); + write_agent_for_host(&catalog, "remote", "worker", false); + fs::create_dir_all(catalog.join("agents/remote/worker/resources/inbox")).unwrap(); + let trap = agent_dir(&catalog, "trap"); + fs::create_dir_all(&trap).unwrap(); + let external = temp.path().join("external-state"); + fs::create_dir(&external).unwrap(); + std::os::unix::fs::symlink(&external, trap.join("resources")).unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + write_agent(&prepared, "new", false); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "marker-created", + &ready, + &release, + ); + wait_for(&ready); + + let existing = send(&catalog, "host.old", "during apply"); + assert!( + existing.status.success(), + "{}", + String::from_utf8_lossy(&existing.stderr) + ); + assert!( + old.join("resources/inbox") + .read_dir() + .unwrap() + .next() + .is_some() + ); + let hierarchical = send(&catalog, "mix.sup", "hierarchical"); + assert!( + hierarchical.status.success(), + "{}", + String::from_utf8_lossy(&hierarchical.stderr) + ); + assert!( + agent_dir(&catalog, "mix.sup") + .join("resources/inbox") + .read_dir() + .unwrap() + .next() + .is_some() + ); + let ambiguous = send(&catalog, "remote.worker", "ambiguous"); + assert!(!ambiguous.status.success()); + let trapped = send(&catalog, "trap", "must not escape"); + assert!(!trapped.status.success()); + assert!(external.read_dir().unwrap().next().is_none()); + + let catalog_text = catalog.to_str().unwrap(); + let context = run_with_stdin( + &[ + "context", + "write", + "host.old", + "--catalog", + catalog_text, + "--host", + "host", + ], + "working during apply", + ); + assert!( + context.status.success(), + "{}", + String::from_utf8_lossy(&context.stderr) + ); + let resource = st2() + .args([ + "resource", + "add", + "https://example.test/result", + "--catalog", + catalog_text, + "--as", + "host.old", + "--host", + "host", + ]) + .output() + .unwrap(); + assert!( + resource.status.success(), + "{}", + String::from_utf8_lossy(&resource.stderr) + ); + let status = st2() + .args([ + "status", + "host.old", + "--set", + "busy", + "--catalog", + catalog_text, + "--host", + "host", + ]) + .output() + .unwrap(); + assert!( + status.status.success(), + "{}", + String::from_utf8_lossy(&status.stderr) + ); + assert_eq!( + fs::read_to_string(old.join("resources/context/now.md")).unwrap(), + "working during apply" + ); + assert!( + old.join("resources/links") + .read_dir() + .unwrap() + .next() + .is_some() + ); + assert_eq!(fs::read_to_string(old.join("status")).unwrap(), "busy\n"); + + let phantom = send(&catalog, "host.new", "too early"); + assert!(!phantom.status.success()); + assert!(!catalog.join("host.new").exists()); + assert!(!agent_dir(&catalog, "new").exists()); + + fs::write(&release, "").unwrap(); + let output = child.wait_with_output().unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(agent_dir(&catalog, "new").join("agent.kdl").is_file()); + + let dotted = temp.path().join("dotted-catalog"); + for path in [ + "agents/a/b.c/resources/inbox", + "agents/a.b/c/resources/inbox", + "agents/a.b/only/resources/inbox", + ] { + fs::create_dir_all(dotted.join(path)).unwrap(); + } + write_test_marker( + &dotted, + &[ + "agents/a/b.c/agent.kdl", + "agents/a.b/c/agent.kdl", + "agents/a.b/only/agent.kdl", + ], + ); + let ambiguous_qualified = send(&dotted, "a.b.c", "ambiguous qualified"); + assert!(!ambiguous_qualified.status.success()); + let dotted_host = send(&dotted, "a.b.only", "dotted host"); + assert!( + dotted_host.status.success(), + "{}", + String::from_utf8_lossy(&dotted_host.stderr) + ); + assert!( + dotted + .join("agents/a.b/only/resources/inbox") + .read_dir() + .unwrap() + .next() + .is_some() + ); +} + +#[test] +fn state_remains_addressable_after_its_spec_is_deleted_mid_apply() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let old = agent_dir(&catalog, "old"); + fs::create_dir_all(old.join("resources/inbox")).unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::remove_file(prepared.join("agents/host/old/agent.kdl")).unwrap(); + fs::remove_dir_all(prepared.join("agents")).unwrap(); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "deleted-spec", + &ready, + &release, + ); + wait_for(&ready); + assert!(!old.join("agent.kdl").exists()); + + let sent = send(&catalog, "host.old", "after delete"); + assert!( + sent.status.success(), + "{}", + String::from_utf8_lossy(&sent.stderr) + ); + assert!( + old.join("resources/inbox") + .read_dir() + .unwrap() + .next() + .is_some() + ); + assert!(!catalog.join("host.old").exists()); + let filename = String::from_utf8(sent.stdout).unwrap(); + let thread = st2() + .args([ + "message", + "thread", + filename.trim(), + "--catalog", + catalog.to_str().unwrap(), + "--host", + "host", + "--tree", + ]) + .output() + .unwrap(); + assert!( + thread.status.success(), + "{}", + String::from_utf8_lossy(&thread.stderr) + ); + assert!(String::from_utf8_lossy(&thread.stdout).contains(filename.trim())); + + fs::write(&release, "").unwrap(); + let output = child.wait_with_output().unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); +} + +#[test] +fn marker_time_message_write_remains_bound_to_its_retained_agent_capability() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + fs::create_dir_all(agent_dir(&catalog, "old").join("resources/inbox")).unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/old/agent.kdl"), + agent("old", true), + ) + .unwrap(); + let apply_ready = temp.path().join("apply-ready"); + let apply_release = temp.path().join("apply-release"); + let apply_child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "marker-created", + &apply_ready, + &apply_release, + ); + wait_for(&apply_ready); + + let message_ready = temp.path().join("message-ready"); + let message_release = temp.path().join("message-release"); + let message = st2() + .args([ + "message", + "send", + "host.old", + "--message", + "capability-bound", + "--as", + "host.sender", + "--host", + "host", + "--catalog", + catalog.to_str().unwrap(), + ]) + .env("ST2_TEST_MESSAGE_CAPABILITY_READY", &message_ready) + .env("ST2_TEST_MESSAGE_CAPABILITY_RELEASE", &message_release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&message_ready); + let retained_host = temp.path().join("retained-host"); + fs::rename(catalog.join("agents/host"), &retained_host).unwrap(); + let outside = temp.path().join("outside-host"); + fs::create_dir_all(outside.join("old/resources/inbox")).unwrap(); + std::os::unix::fs::symlink(&outside, catalog.join("agents/host")).unwrap(); + fs::write(&message_release, "").unwrap(); + let message = message.wait_with_output().unwrap(); + assert!( + message.status.success(), + "{}", + String::from_utf8_lossy(&message.stderr) + ); + assert!( + retained_host + .join("old/resources/inbox") + .read_dir() + .unwrap() + .next() + .is_some() + ); + assert!( + outside + .join("old/resources/inbox") + .read_dir() + .unwrap() + .next() + .is_none() + ); + fs::remove_file(catalog.join("agents/host")).unwrap(); + fs::rename(&retained_host, catalog.join("agents/host")).unwrap(); + fs::write(&apply_release, "").unwrap(); + let applied = apply_child.wait_with_output().unwrap(); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); +} + +#[test] +fn marker_time_status_write_remains_bound_to_its_retained_agent_capability() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/old/agent.kdl"), + agent("old", true), + ) + .unwrap(); + let apply_ready = temp.path().join("apply-ready"); + let apply_release = temp.path().join("apply-release"); + let apply_child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "marker-created", + &apply_ready, + &apply_release, + ); + wait_for(&apply_ready); + + let state_ready = temp.path().join("state-ready"); + let state_release = temp.path().join("state-release"); + let state = st2() + .args([ + "status", + "host.old", + "--set", + "busy", + "--host", + "host", + "--catalog", + catalog.to_str().unwrap(), + ]) + .env("ST2_TEST_MESSAGE_CAPABILITY_READY", &state_ready) + .env("ST2_TEST_MESSAGE_CAPABILITY_RELEASE", &state_release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&state_ready); + let retained_host = temp.path().join("retained-host"); + fs::rename(catalog.join("agents/host"), &retained_host).unwrap(); + let outside = temp.path().join("outside-host"); + fs::create_dir_all(outside.join("old")).unwrap(); + std::os::unix::fs::symlink(&outside, catalog.join("agents/host")).unwrap(); + fs::write(&state_release, "").unwrap(); + let state = state.wait_with_output().unwrap(); + assert!( + state.status.success(), + "{}", + String::from_utf8_lossy(&state.stderr) + ); + assert_eq!( + fs::read_to_string(retained_host.join("old/status")).unwrap(), + "busy\n" + ); + assert!(!outside.join("old/status").exists()); + fs::remove_file(catalog.join("agents/host")).unwrap(); + fs::rename(&retained_host, catalog.join("agents/host")).unwrap(); + fs::write(&apply_release, "").unwrap(); + let applied = apply_child.wait_with_output().unwrap(); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); +} + +#[test] +fn marker_time_context_and_resource_writes_reject_a_swapped_state_ancestor() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + fs::create_dir_all(agent_dir(&catalog, "old").join("resources")).unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/old/agent.kdl"), + agent("old", true), + ) + .unwrap(); + let apply_ready = temp.path().join("apply-ready"); + let apply_release = temp.path().join("apply-release"); + let apply_child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "marker-created", + &apply_ready, + &apply_release, + ); + wait_for(&apply_ready); + + for (name, args, input) in [ + ( + "context", + vec![ + "context", + "write", + "host.old", + "--catalog", + catalog.to_str().unwrap(), + "--host", + "host", + ], + "must-not-land", + ), + ( + "resource", + vec![ + "resource", + "add", + "https://example.test/must-not-land", + "--catalog", + catalog.to_str().unwrap(), + "--as", + "host.old", + "--host", + "host", + ], + "", + ), + ] { + let ready = temp.path().join(format!("{name}-ready")); + let release = temp.path().join(format!("{name}-release")); + let mut child = st2() + .args(args) + .env("ST2_TEST_MESSAGE_CAPABILITY_READY", &ready) + .env("ST2_TEST_MESSAGE_CAPABILITY_RELEASE", &release) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + child + .stdin + .take() + .unwrap() + .write_all(input.as_bytes()) + .unwrap(); + wait_for(&ready); + let resources = agent_dir(&catalog, "old").join("resources"); + let retained = temp.path().join(format!("{name}-retained-resources")); + fs::rename(&resources, &retained).unwrap(); + let outside = temp.path().join(format!("{name}-outside")); + fs::create_dir(&outside).unwrap(); + std::os::unix::fs::symlink(&outside, &resources).unwrap(); + fs::write(&release, "").unwrap(); + let output = child.wait_with_output().unwrap(); + assert!(!output.status.success()); + assert!(outside.read_dir().unwrap().next().is_none()); + fs::remove_file(&resources).unwrap(); + fs::rename(retained, resources).unwrap(); + } + + fs::write(&apply_release, "").unwrap(); + let applied = apply_child.wait_with_output().unwrap(); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); +} + +#[test] +fn completed_apply_between_address_fence_reads_cannot_accept_a_stale_recipient() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "old", false); + fs::create_dir_all(agent_dir(&catalog, "old").join("resources/inbox")).unwrap(); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::remove_dir_all(prepared.join("agents")).unwrap(); + let ready = temp.path().join("fence-ready"); + let release = temp.path().join("fence-release"); + let message = st2() + .args([ + "message", + "send", + "host.old", + "--message", + "must-not-land", + "--as", + "host.sender", + "--host", + "host", + "--catalog", + catalog.to_str().unwrap(), + ]) + .env("ST2_TEST_ADDRESS_FENCE_READY", &ready) + .env("ST2_TEST_ADDRESS_FENCE_RELEASE", &release) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&ready); + let applied = apply(&catalog, &prepared, before["rootSha256"].as_str().unwrap()); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); + fs::write(&release, "").unwrap(); + let message = message.wait_with_output().unwrap(); + assert!(!message.status.success()); + assert!( + agent_dir(&catalog, "old") + .join("resources/inbox") + .read_dir() + .unwrap() + .next() + .is_none() + ); + assert!(!catalog.join("host.old").exists()); +} + +#[test] +fn task_inventory_fails_closed_without_observing_runtime_during_a_partial_apply() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "a", false); + write_agent(&catalog, "b", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write(prepared.join("agents/host/a/agent.kdl"), agent("a", true)).unwrap(); + fs::write(prepared.join("agents/host/b/agent.kdl"), agent("b", true)).unwrap(); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "mid-write", + &ready, + &release, + ); + wait_for(&ready); + let changed = ["a", "b"] + .iter() + .filter(|identity| { + fs::read_to_string(agent_dir(&catalog, identity).join("agent.kdl")) + .unwrap() + .contains("retired #true") + }) + .count(); + assert_eq!(changed, 1, "fixture must expose a stable partial catalog"); + + let bin = temp.path().join("bin"); + fs::create_dir(&bin).unwrap(); + let sentinel = temp.path().join("runtime-observed"); + let pty = bin.join("pty"); + fs::write( + &pty, + format!("#!/bin/sh\n: > {:?}\nprintf '[]\\n'\n", sentinel), + ) + .unwrap(); + let mut permissions = fs::metadata(&pty).unwrap().permissions(); + use std::os::unix::fs::PermissionsExt as _; + permissions.set_mode(0o755); + fs::set_permissions(&pty, permissions).unwrap(); + let tasks = st2() + .args([ + "tasks", + "--host", + "host", + "--json", + "--catalog", + catalog.to_str().unwrap(), + ]) + .env("PATH", &bin) + .output() + .unwrap(); + assert!(!tasks.status.success()); + let inventory: Value = serde_json::from_slice(&tasks.stdout).unwrap(); + assert_eq!(inventory["complete"], false); + assert!(inventory["tasks"].as_array().unwrap().is_empty()); + assert!(inventory["errors"].as_array().unwrap().iter().any(|error| { + error + .as_str() + .is_some_and(|error| error.contains("apply is incomplete")) + })); + assert!( + !sentinel.exists(), + "runtime observer ran inside the fenced view" + ); + + fs::write(&release, "").unwrap(); + let applied = child.wait_with_output().unwrap(); + assert!( + applied.status.success(), + "{}", + String::from_utf8_lossy(&applied.stderr) + ); +} + +#[test] +fn agent_publish_serializes_behind_apply_and_rechecks_its_leaf_cas() { + let temp = tempfile::tempdir().unwrap(); + let catalog = temp.path().join("catalog"); + write_agent(&catalog, "worker", false); + let old = agent("worker", false); + let prepared = temp.path().join("prepared"); + let before = snapshot(&catalog, &prepared); + fs::write( + prepared.join("agents/host/worker/agent.kdl"), + agent("worker", true), + ) + .unwrap(); + let publisher_spec = temp.path().join("publisher.kdl"); + fs::write( + &publisher_spec, + "agent \"worker\" {\n host \"host\"\n role \"publisher\"\n argv \"true\"\n}\n", + ) + .unwrap(); + let digest = st2() + .args(["agent", "digest", "--spec"]) + .arg(&publisher_spec) + .arg("--json") + .output() + .unwrap(); + assert!(digest.status.success()); + let digest: Value = serde_json::from_slice(&digest.stdout).unwrap(); + let ready = temp.path().join("ready"); + let release = temp.path().join("release"); + let apply_child = paused_apply( + &catalog, + &prepared, + before["rootSha256"].as_str().unwrap(), + "marker-created", + &ready, + &release, + ); + wait_for(&ready); + + let mut publisher = st2() + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + publisher_spec.to_str().unwrap(), + "--input-sha256", + digest["sha256"].as_str().unwrap(), + "--expect-sha256", + &format!("{:x}", Sha256::digest(old.as_bytes())), + "--json", + ]) + .env( + "ST2_TEST_CATALOG_LOCK_ATTEMPT", + temp.path().join("publisher-waiting"), + ) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + wait_for(&temp.path().join("publisher-waiting")); + assert!(publisher.try_wait().unwrap().is_none()); + + fs::write(&release, "").unwrap(); + let apply_output = apply_child.wait_with_output().unwrap(); + assert!( + apply_output.status.success(), + "{}", + String::from_utf8_lossy(&apply_output.stderr) + ); + let publisher = publisher.wait_with_output().unwrap(); + assert!(!publisher.status.success()); + assert!( + String::from_utf8_lossy(&publisher.stderr).contains("precondition failed"), + "{}", + String::from_utf8_lossy(&publisher.stderr) + ); + assert_eq!( + fs::read_to_string(agent_dir(&catalog, "worker").join("agent.kdl")).unwrap(), + agent("worker", true) + ); +} diff --git a/tests/catalog_selection.rs b/tests/catalog_selection.rs index 04bedcaa..6ea66fda 100644 --- a/tests/catalog_selection.rs +++ b/tests/catalog_selection.rs @@ -6,6 +6,8 @@ use std::fs; use std::path::Path; use std::process::{Command, Output}; +use sha2::{Digest as _, Sha256}; + fn write_agent(catalog: &Path, host: &str, identity: &str) { let dir = catalog.join("agents").join(host).join(identity); fs::create_dir_all(&dir).unwrap(); @@ -118,12 +120,14 @@ fn agent_publish_can_target_only_the_global_catalog_flag() { "agent \"worker\" {\n host \"h\"\n argv \"true\"\n}\n", ) .unwrap(); + let input_sha256 = format!("{:x}", Sha256::digest(fs::read(&spec).unwrap())); let out = Command::new(env!("CARGO_BIN_EXE_st2")) .arg("--catalog") .arg(&catalog) .args(["agent", "publish", "--spec"]) .arg(&spec) + .args(["--input-sha256", &input_sha256]) .arg("--expect-absent") .output() .unwrap(); diff --git a/tests/materialize.rs b/tests/materialize.rs index c47cb453..e42c851d 100644 --- a/tests/materialize.rs +++ b/tests/materialize.rs @@ -11,9 +11,8 @@ fn write(path: &Path, contents: impl AsRef<[u8]>) { } #[test] -fn task_selector_refusal_is_nonzero_before_catalog_mutation() { +fn task_selector_refusal_initializes_only_the_persistent_coordination_lock() { let tmp = tempfile::tempdir().unwrap(); - let before = fs::read_dir(tmp.path()).unwrap().count(); let out = Command::new(env!("CARGO_BIN_EXE_st2")) .args(["up", "--catalog"]) .arg(tmp.path()) @@ -27,7 +26,20 @@ fn task_selector_refusal_is_nonzero_before_catalog_mutation() { .output() .unwrap(); assert!(!out.status.success()); - assert_eq!(fs::read_dir(tmp.path()).unwrap().count(), before); + let root_entries = fs::read_dir(tmp.path()) + .unwrap() + .collect::, _>>() + .unwrap(); + assert_eq!(root_entries.len(), 1); + assert_eq!(root_entries[0].file_name(), ".st2"); + let control_entries = fs::read_dir(root_entries[0].path()) + .unwrap() + .collect::, _>>() + .unwrap(); + assert_eq!(control_entries.len(), 1); + assert_eq!(control_entries[0].file_name(), "catalog-authoring.lock"); + let lock = fs::symlink_metadata(control_entries[0].path()).unwrap(); + assert!(lock.is_file() && !lock.file_type().is_symlink()); } #[test] diff --git a/tests/message.rs b/tests/message.rs index 23bf32dc..44dc8009 100644 --- a/tests/message.rs +++ b/tests/message.rs @@ -37,12 +37,24 @@ fn agent_kdl(identity: &str, host: &str) -> String { fn send_by_bus_id_lands_in_recipient_inbox() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - write(root, "hetz/st2-claude/agent.kdl", &agent_kdl("st2-claude", "hetz")); - write(root, "hetz/cos-claude/agent.kdl", &agent_kdl("cos-claude", "hetz")); + write( + root, + "hetz/st2-claude/agent.kdl", + &agent_kdl("st2-claude", "hetz"), + ); + write( + root, + "hetz/cos-claude/agent.kdl", + &agent_kdl("cos-claude", "hetz"), + ); // Resolve the recipient's agent folder by its bus id, then by bare identity — both must match. - let dir_by_bus = resolve_agent_dir(root, "hetz.st2-claude", "hetz").expect("resolve by bus id"); - let dir_by_ident = resolve_agent_dir(root, "st2-claude", "hetz").expect("resolve by identity"); + let dir_by_bus = resolve_agent_dir(root, "hetz.st2-claude", "hetz") + .unwrap() + .expect("resolve by bus id"); + let dir_by_ident = resolve_agent_dir(root, "st2-claude", "hetz") + .unwrap() + .expect("resolve by identity"); assert_eq!(dir_by_bus, dir_by_ident); assert_eq!(dir_by_bus, root.join("hetz/st2-claude")); @@ -51,7 +63,10 @@ fn send_by_bus_id_lands_in_recipient_inbox() { let f = send_to_inbox(&inbox, "hetz.cos-claude", Some("kick"), None, &[], "do M2").unwrap(); assert!(inbox.join(&f).exists()); let other = inbox_dir(&root.join("hetz/cos-claude")); - assert!(list_dir(&other).unwrap().is_empty(), "must not leak into the other agent's inbox"); + assert!( + list_dir(&other).unwrap().is_empty(), + "must not leak into the other agent's inbox" + ); let listed = list_dir(&inbox).unwrap(); assert_eq!(listed.len(), 1); @@ -69,8 +84,30 @@ fn send_by_bus_id_lands_in_recipient_inbox() { fn unknown_recipient_does_not_resolve() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - write(root, "hetz/st2-claude/agent.kdl", &agent_kdl("st2-claude", "hetz")); - assert!(resolve_agent_dir(root, "hetz.nobody", "hetz").is_none()); + write( + root, + "hetz/st2-claude/agent.kdl", + &agent_kdl("st2-claude", "hetz"), + ); + assert!( + resolve_agent_dir(root, "hetz.nobody", "hetz") + .unwrap() + .is_none() + ); +} + +#[test] +fn archive_rejects_noncanonical_leaf_paths() { + let tmp = tempfile::tempdir().unwrap(); + let inbox = tmp.path().join("inbox"); + let archive = tmp.path().join("archive"); + fs::create_dir_all(&inbox).unwrap(); + fs::write(tmp.path().join("outside.md"), "unchanged").unwrap(); + assert!(archive_msg(&inbox, &archive, "../outside.md").is_err()); + assert_eq!( + fs::read_to_string(tmp.path().join("outside.md")).unwrap(), + "unchanged" + ); } /// `message thread` walks the reply chain ACROSS agents — a two-party conversation lives in both @@ -80,29 +117,76 @@ fn unknown_recipient_does_not_resolve() { fn thread_walks_the_reply_chain_across_both_agents() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - write(root, "h/alice-claude/agent.kdl", &agent_kdl("alice-claude", "h")); - write(root, "h/bob-claude/agent.kdl", &agent_kdl("bob-claude", "h")); - let alice = resolve_agent_dir(root, "h.alice-claude", "h").unwrap(); - let bob = resolve_agent_dir(root, "h.bob-claude", "h").unwrap(); + write( + root, + "h/alice-claude/agent.kdl", + &agent_kdl("alice-claude", "h"), + ); + write( + root, + "h/bob-claude/agent.kdl", + &agent_kdl("bob-claude", "h"), + ); + let alice = resolve_agent_dir(root, "h.alice-claude", "h") + .unwrap() + .unwrap(); + let bob = resolve_agent_dir(root, "h.bob-claude", "h") + .unwrap() + .unwrap(); // alice → bob (root), bob → alice (reply), alice → bob (reply-to-reply). - let f1 = send_to_inbox(&inbox_dir(&bob), "h.alice-claude", Some("plan"), None, &[], "kickoff").unwrap(); + let f1 = send_to_inbox( + &inbox_dir(&bob), + "h.alice-claude", + Some("plan"), + None, + &[], + "kickoff", + ) + .unwrap(); std::thread::sleep(std::time::Duration::from_millis(2)); - let f2 = send_to_inbox(&inbox_dir(&alice), "h.bob-claude", Some("re: plan"), Some(&f1), &[], "on it").unwrap(); + let f2 = send_to_inbox( + &inbox_dir(&alice), + "h.bob-claude", + Some("re: plan"), + Some(&f1), + &[], + "on it", + ) + .unwrap(); std::thread::sleep(std::time::Duration::from_millis(2)); - let f3 = send_to_inbox(&inbox_dir(&bob), "h.alice-claude", Some("re: plan"), Some(&f2), &[], "go").unwrap(); + let f3 = send_to_inbox( + &inbox_dir(&bob), + "h.alice-claude", + Some("re: plan"), + Some(&f2), + &[], + "go", + ) + .unwrap(); // From ANY member, the whole thread comes back, root-first, in reply order. for start in [&f1, &f2, &f3] { - let thread = collect_thread(root, start); + let thread = collect_thread(root, start).unwrap(); let names: Vec<&str> = thread.iter().map(|e| e.filename.as_str()).collect(); - assert_eq!(names, [f1.as_str(), f2.as_str(), f3.as_str()], "thread from {start}"); + assert_eq!( + names, + [f1.as_str(), f2.as_str(), f3.as_str()], + "thread from {start}" + ); // Depth increases down the reply chain. - assert_eq!(thread.iter().map(|e| e.depth).collect::>(), [0, 1, 2]); + assert_eq!( + thread.iter().map(|e| e.depth).collect::>(), + [0, 1, 2] + ); } // An unknown filename → empty thread. - assert!(collect_thread(root, "0000000000000-nope00.md").is_empty()); + assert!( + collect_thread(root, "0000000000000-nope00.md") + .unwrap() + .is_empty() + ); } /// The reply flow: read an inbound message, derive recipient (its `from`) + threading, and land the @@ -111,15 +195,34 @@ fn thread_walks_the_reply_chain_across_both_agents() { fn reply_threads_back_to_the_original_sender() { let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - write(root, "hetz/st2-claude/agent.kdl", &agent_kdl("st2-claude", "hetz")); - write(root, "hetz/cos-claude/agent.kdl", &agent_kdl("cos-claude", "hetz")); + write( + root, + "hetz/st2-claude/agent.kdl", + &agent_kdl("st2-claude", "hetz"), + ); + write( + root, + "hetz/cos-claude/agent.kdl", + &agent_kdl("cos-claude", "hetz"), + ); - let me = resolve_agent_dir(root, "hetz.st2-claude", "hetz").unwrap(); - let cos = resolve_agent_dir(root, "hetz.cos-claude", "hetz").unwrap(); + let me = resolve_agent_dir(root, "hetz.st2-claude", "hetz") + .unwrap() + .unwrap(); + let cos = resolve_agent_dir(root, "hetz.cos-claude", "hetz") + .unwrap() + .unwrap(); // cos → me - let original = - send_to_inbox(&inbox_dir(&me), "hetz.cos-claude", Some("M2 kick"), None, &[], "go").unwrap(); + let original = send_to_inbox( + &inbox_dir(&me), + "hetz.cos-claude", + Some("M2 kick"), + None, + &[], + "go", + ) + .unwrap(); // me replies: recipient + subject + in-reply-to derived from the original in my inbox. let inbound = read_msg(&inbox_dir(&me), &original).unwrap(); @@ -179,15 +282,7 @@ fn a_concurrent_reader_never_observes_a_half_written_message() { let inbox = root.join(format!("inbox-{i}")); fs::create_dir_all(&inbox).unwrap(); cursor.store(i, Ordering::Relaxed); - send_to_inbox( - &inbox, - "alice", - Some(&format!("big {i}")), - None, - &[], - &body, - ) - .unwrap(); + send_to_inbox(&inbox, "alice", Some(&format!("big {i}")), None, &[], &body).unwrap(); } done.store(true, Ordering::Relaxed); diff --git a/tests/native_only.rs b/tests/native_only.rs index 78d4317e..afa8217a 100644 --- a/tests/native_only.rs +++ b/tests/native_only.rs @@ -108,11 +108,26 @@ fn clean_path_executes_the_maintained_native_authoring_guide() { } for bundle in [&codex_bundle, &claude_bundle] { + let digest = clean_st2(bin.path(), &state, &hooks) + .args(["agent", "digest", "--bundle"]) + .arg(bundle) + .arg("--json") + .output() + .unwrap(); + assert!( + digest.status.success(), + "stdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&digest.stdout), + String::from_utf8_lossy(&digest.stderr) + ); + let digest: serde_json::Value = serde_json::from_slice(&digest.stdout).unwrap(); let publish = clean_st2(bin.path(), &state, &hooks) .args(["agent", "publish", "--catalog"]) .arg(&catalog) .arg("--bundle") .arg(bundle) + .arg("--input-sha256") + .arg(digest["sha256"].as_str().unwrap()) .args(["--expect-absent", "--json"]) .output() .unwrap(); diff --git a/tests/task_inventory_cli.rs b/tests/task_inventory_cli.rs index c32c6dee..6159419a 100644 --- a/tests/task_inventory_cli.rs +++ b/tests/task_inventory_cli.rs @@ -117,6 +117,239 @@ fn tasks_cli_emits_stable_complete_generation_without_mutation() { assert!(!state.exists(), "read-only inventory created runtime state"); } +#[test] +fn completed_catalog_aba_during_runtime_observation_is_incomplete() { + let (tmp, catalog, bin) = fixture("[]"); + let prepared_a = tmp.path().join("prepared-a"); + let prepared_b = tmp.path().join("prepared-b"); + let snapshot = |output: &Path| { + let result = Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "catalog", + "snapshot", + "--catalog", + catalog.to_str().unwrap(), + "--output", + output.to_str().unwrap(), + "--json", + ]) + .output() + .unwrap(); + assert!( + result.status.success(), + "{}", + String::from_utf8_lossy(&result.stderr) + ); + serde_json::from_slice::(&result.stdout).unwrap() + }; + let root_a = snapshot(&prepared_a)["rootSha256"] + .as_str() + .unwrap() + .to_string(); + snapshot(&prepared_b); + let b_spec = prepared_b.join("agents/h/worker/agent.kdl"); + let bytes = fs::read_to_string(&b_spec) + .unwrap() + .replace("agent \"worker\" {", "agent \"worker\" {\n retired #true"); + fs::write(&b_spec, bytes).unwrap(); + + let observer_ready = tmp.path().join("observer-ready"); + let observer_release = tmp.path().join("observer-release"); + write_executable( + &bin.join("pty"), + &format!( + "#!/bin/sh\n: > {:?}\nwhile [ ! -e {:?} ]; do :; done\nprintf '[]\\n'\n", + observer_ready, observer_release + ), + ); + let inventory = Command::new(env!("CARGO_BIN_EXE_st2")) + .args(["tasks", "--host", "h", "--json", "--catalog"]) + .arg(&catalog) + .env("PATH", &bin) + .env("XDG_STATE_HOME", tmp.path().join("state")) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + while !observer_ready.exists() { + assert!( + std::time::Instant::now() < deadline, + "runtime observer did not start" + ); + std::thread::yield_now(); + } + + let apply = |prepared: &Path, expected: &str| { + Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "catalog", + "apply", + "--catalog", + catalog.to_str().unwrap(), + "--prepared", + prepared.to_str().unwrap(), + "--expect-sha256", + expected, + "--json", + ]) + .output() + .unwrap() + }; + let to_b = apply(&prepared_b, &root_a); + assert!( + to_b.status.success(), + "{}", + String::from_utf8_lossy(&to_b.stderr) + ); + let root_b = serde_json::from_slice::(&to_b.stdout).unwrap()["afterSha256"] + .as_str() + .unwrap() + .to_string(); + let to_a = apply(&prepared_a, &root_b); + assert!( + to_a.status.success(), + "{}", + String::from_utf8_lossy(&to_a.stderr) + ); + fs::write(&observer_release, "").unwrap(); + + let inventory = inventory.wait_with_output().unwrap(); + assert!(!inventory.status.success()); + let value: serde_json::Value = serde_json::from_slice(&inventory.stdout).unwrap(); + assert_eq!(value["complete"], false); + assert!(value["errors"].as_array().unwrap().iter().any(|error| { + error + .as_str() + .is_some_and(|error| error.contains("generation changed")) + })); +} + +#[test] +fn completed_single_agent_writer_abas_during_runtime_observation_are_incomplete() { + for writer in ["publish", "presentation"] { + let (tmp, catalog, bin) = fixture("[]"); + let spec = catalog.join("agents/h/worker/agent.kdl"); + let original = fs::read_to_string(&spec).unwrap(); + let observer_ready = tmp.path().join("observer-ready"); + let observer_release = tmp.path().join("observer-release"); + write_executable( + &bin.join("pty"), + &format!( + "#!/bin/sh\n: > {:?}\nwhile [ ! -e {:?} ]; do :; done\nprintf '[]\\n'\n", + observer_ready, observer_release + ), + ); + let inventory = Command::new(env!("CARGO_BIN_EXE_st2")) + .args(["tasks", "--host", "h", "--json", "--catalog"]) + .arg(&catalog) + .env("PATH", &bin) + .env("XDG_STATE_HOME", tmp.path().join("state")) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + let deadline = std::time::Instant::now() + Duration::from_secs(5); + while !observer_ready.exists() { + assert!( + std::time::Instant::now() < deadline, + "runtime observer did not start" + ); + std::thread::yield_now(); + } + + if writer == "publish" { + let source_a = tmp.path().join("source-a.kdl"); + let source_b = tmp.path().join("source-b.kdl"); + fs::write(&source_a, &original).unwrap(); + fs::write( + &source_b, + original.replace( + "agent \"worker\" {", + "agent \"worker\" {\n name \"temporary\"", + ), + ) + .unwrap(); + let digest = |source: &Path| { + let output = Command::new(env!("CARGO_BIN_EXE_st2")) + .args(["agent", "digest", "--spec"]) + .arg(source) + .arg("--json") + .output() + .unwrap(); + assert!(output.status.success()); + serde_json::from_slice::(&output.stdout).unwrap()["sha256"] + .as_str() + .unwrap() + .to_string() + }; + let digest_a = digest(&source_a); + let digest_b = digest(&source_b); + for (source, input, expected) in [ + (&source_b, digest_b.as_str(), digest_a.as_str()), + (&source_a, digest_a.as_str(), digest_b.as_str()), + ] { + let output = Command::new(env!("CARGO_BIN_EXE_st2")) + .args([ + "agent", + "publish", + "--catalog", + catalog.to_str().unwrap(), + "--spec", + source.to_str().unwrap(), + "--input-sha256", + input, + "--expect-sha256", + expected, + ]) + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + } + } else { + for args in [ + vec!["rename", "h.worker", "temporary", "--host", "h"], + vec!["rename", "h.worker", "--clear", "--host", "h"], + ] { + let output = Command::new(env!("CARGO_BIN_EXE_st2")) + .arg("--catalog") + .arg(&catalog) + .args(args) + .env_remove("ST_AGENT") + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + } + } + assert_eq!(fs::read_to_string(&spec).unwrap(), original); + assert_eq!( + fs::read_to_string(catalog.join(".st2/catalog-generation")).unwrap(), + "2\n" + ); + fs::write(&observer_release, "").unwrap(); + let inventory = inventory.wait_with_output().unwrap(); + assert!( + !inventory.status.success(), + "{writer} ABA was reported complete" + ); + let value: serde_json::Value = serde_json::from_slice(&inventory.stdout).unwrap(); + assert_eq!(value["complete"], false); + assert!(value["errors"].as_array().unwrap().iter().any(|error| { + error + .as_str() + .is_some_and(|error| error.contains("generation changed")) + })); + } +} + #[test] fn incomplete_or_malformed_pty_generation_is_json_nonzero_and_never_absent() { for pty_json in [ diff --git a/tests/validate.rs b/tests/validate.rs index 0b136445..931332a3 100644 --- a/tests/validate.rs +++ b/tests/validate.rs @@ -151,6 +151,77 @@ fn a_relative_path_is_an_error() { assert!(has(&validate(c.path()), "bad-path", Severity::Error)); } +#[test] +fn canonical_relative_workspace_and_task_cwd_are_clean() { + let c = catalog(&[ + ( + "hetz/w/agent.kdl", + r#"agent "w" { + host "hetz" + workspace ".workspace" + pty "agent" { cwd ".workspace"; command "x" } +}"#, + ), + ("hetz/w/.workspace/.keep", ""), + ]); + assert!(!has(&validate(c.path()), "bad-path", Severity::Error)); +} + +#[test] +fn normalized_relative_workspace_is_clean_but_indeterminate_relative_fails_closed() { + let temp = tempfile::tempdir().unwrap(); + for (identity, workspace) in [ + ("dotted", "./.workspace"), + ("indeterminate", "$ST2_TEST_UNSET_WORKSPACE"), + ] { + let bundle = temp.path().join(format!("agents/host/{identity}")); + std::fs::create_dir_all(bundle.join(".workspace")).unwrap(); + std::fs::write( + bundle.join("agent.kdl"), + format!( + "agent \"{identity}\" {{\n host \"host\"\n workspace \"{workspace}\"\n argv \"true\"\n}}\n" + ), + ) + .unwrap(); + } + + let report = validate(temp.path()); + let bad_paths = report + .issues + .iter() + .filter(|issue| issue.code == "bad-path") + .collect::>(); + assert_eq!(bad_paths.len(), 1, "{:#?}", report.issues); + assert!( + bad_paths + .iter() + .all(|issue| issue.message.contains("unresolved environment variable")) + ); +} + +#[test] +fn environment_expanded_canonical_relative_workspace_is_clean() { + let c = catalog(&[ + ( + "hetz/w/agent.kdl", + r#"agent "w" { host "hetz"; workspace "$ST2_TEST_WORKSPACE"; argv "true" }"#, + ), + ("hetz/w/.workspace/.keep", ""), + ]); + let output = std::process::Command::new(env!("CARGO_BIN_EXE_st2")) + .arg("--catalog") + .arg(c.path()) + .args(["validate", "--json"]) + .env("ST2_TEST_WORKSPACE", ".workspace") + .output() + .unwrap(); + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stdout) + ); +} + #[test] fn a_missing_catalog_rooted_path_is_an_error() { // The renderer's own output — its absence is a real render bug. @@ -196,8 +267,7 @@ fn a_remote_hosts_missing_external_path_is_not_a_local_warning() { } #[test] -fn a_path_bearing_another_var_is_skipped() { - // SD3: an unset var is a literal token — do not guess, do not flag. +fn an_environment_expanded_absolute_path_is_checked_normally() { let c = catalog(&[( "hetz/w/agent.kdl", r#"agent "w" { host "hetz"; type "service"; workspace "$HOME/repo"; pty "agent" { command "x" } }"#,