feat: publish agent specs transactionally

agent-session-id: dev3.dotfiles-cos-misc-agent-runtime-simplification
agent-tool: Codex
agent-tool-version: 0.145.0
agent-model: gpt-5.6-sol
agent-runtime-profile: /home/schickling/.config/coding-agents/profile.json
agent-skills-manifest: /nix/store/nk9iml2841l1yjjg0f6f0d3y60zkg1nn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@4a0515f

Author: schickling-assistant

INVARIANTS.md

@@ -13,7 +13,7 @@ materialization, messaging, DING, or presence must preserve them.
 | **Fail-closed observed native DING** | Each unread message becomes one normalized `[DING]` frame. Fresh delivery records ownership, then preserves the one combined bracketed-paste, 0.5 second delay, and Return transaction. PTY and Return success are transport only: `Delivered` additionally requires adapter classification of the expected notice text in a submitted-prompt or queued-message pattern while the lowest live composer is empty or an accepted idle placeholder. Every other receipt retains staged ownership. Retry never re-pastes and may send one bare Return only after two adjacent `RetainedSafe` observations; human, modal, active, changed, timed-out, and unknown retry states receive no input. Ownership prevents duplicate paste across command failures, receipt ambiguity, archive races, and restart adoption. Startup backlog otherwise becomes one generic recovery DING; new arrivals remain FIFO; `busy` delivers immediately; only fresh `dnd` defers. | `src/ding/mod.rs::poke_text_normalizes_and_bounds_untrusted_fields`; `src/ding/mod.rs::malicious_controls_cannot_escape_the_single_paste_frame`; `src/ding/mod.rs::pty_delivery_uses_face607_delay_order_and_seconds`; `src/ding/mod.rs::maintained_composer_classifiers_require_exact_idle_state`; `src/ding/mod.rs::successful_transport_with_retained_or_unproven_pixels_is_not_delivered`; `src/ding/mod.rs::ambiguous_transport_receipt_and_retry_errors_retain_staged_ownership`; `src/ding/mod.rs::adapter_recognized_notice_with_an_empty_live_composer_is_a_positive_receipt`; `src/ding/mod.rs::staged_retry_submits_only_retained_safe_and_requires_a_receipt`; `src/ding/mod.rs::staged_ownership_survives_archive_and_never_repastes`; `src/ding/mod.rs::pty_commands_have_a_real_outer_timeout`; `src/ding/mod.rs::session_watch_has_startup_grace_debounce_and_live_reset`; `src/ding/mod.rs::new_arrivals_is_fifo_and_archive_receipts_prevent_reding`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry`; `src/ding/mod.rs::startup_recovery_notice_retries_in_memory`; `src/ding/mod.rs::startup_backlog_gets_one_generic_recovery_then_new_arrivals_poke` |
 | **Mutation-only filesystem wakeups** | Supervisor and DING filesystem watchers ignore read/open access events and wake early only for create, modify, rename, or remove events. Their own catalog and inbox reads therefore cannot bypass the bounded timer cadence or form a Linux inotify CPU loop. | `src/watch.rs::only_mutations_wake_watch_loops`; `src/watch.rs::linux_reads_are_silent_but_real_mutations_wake`; `src/ding/mod.rs::idle_ding_does_not_spin_on_its_own_inbox_reads`; `src/run.rs::idle_supervisor_does_not_spin_on_its_own_catalog_reads` |
 | **Bounded DING PTY probe churn** | An unsafe or active composer retains its FIFO notice but deferred delivery retries use a bounded backoff, so each inbox poll cannot spawn another short-lived PTY probe. | `src/ding/mod.rs::deferred_delivery_backoff_bounds_short_lived_pty_attempts` |
-| **Agent-declared presence discipline** | The shipped bus contract requires agents to declare `busy` before executing work, use `available` only while yielding or ready, and reserve `dnd` for an explicit hold. Both native harnesses materialize that contract. Busy remains observable but does not suppress DING; fresh `dnd` is the only delivery gate. | `tests/compile_agent.rs::compile_agent_generates_claude_then_materializes_verbatim_persona`; `tests/compile_agent.rs::compile_agent_generates_codex_then_materializes_composed_agents_md`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry` |
+| **Agent-declared presence discipline** | The shipped bus contract requires agents to declare `busy` before executing work, use `available` only while yielding or ready, and reserve `dnd` for an explicit hold. Both native harnesses materialize that contract. Busy remains observable but does not suppress DING; fresh `dnd` is the only delivery gate. | `tests/native_only.rs::clean_path_executes_the_maintained_native_authoring_guide`; `src/ding/mod.rs::pending_delivery_ignores_busy_but_respects_fresh_dnd_archive_and_retry` |
 | **Stable roster JSON** | `st2 agents --json [--enrich]` preserves field names, order, null handling, presence, explicit retirement state, opaque declared Resource descriptors, activity, and inbox counts. Human output marks retired declarations without changing active rows. | `src/agents.rs::agents_json_has_stable_wire_shape`; `src/agents.rs::agents_json_preserves_opaque_declared_resource_descriptors`; `tests/status_agents.rs::roster_json_and_human_output_distinguish_retirement_from_presence` |
 | **Agent-declared presence** | Refresh preserves non-DND declared status and only advances liveness; a missing status starts as `available`, while `dnd` is never refreshed and an unrefreshed declaration ages to `unknown`. | `src/status.rs::refresh_preserves_value_and_bumps_mtime`; `src/status.rs::refresh_leaves_dnd_to_age_out`; `src/status.rs::refresh_missing_writes_available_default`; `src/status.rs::stale_mtime_reads_as_unknown_regardless_of_contents` |
 | **Retirement health** | A retired declaration is healthy only after every declared task ID is absent. Any live or dead declared task record reports incomplete retirement; retired declarations do not require presence. Live declarations retain their existing task and presence checks. | `tests/doctor.rs::retired_declaration_is_healthy_when_tasks_and_presence_are_absent`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_declared_task_is_alive`; `tests/doctor.rs::retired_declaration_is_unhealthy_while_a_dead_task_record_remains` |

README.md

@@ -4,8 +4,8 @@ st2 runs a declarative network of Codex and Claude agents from one catalog. It o
 reconciliation, native messages, normalized terminal DING delivery, presence, durable context,
 workspace materialization, and explicit teardown.
 
-Hand-authored KDL is the canonical interface. `st2 compile-agent` is experimental and must be
-reviewed before its output is materialized.
+Canonical KDL is the authoring interface. Publish one explicit Agent Spec transactionally with
+`st2 agent publish`; st2 does not compile human intent into declarations.
 
 ## Product intent and implementation contract
 
@@ -92,13 +92,17 @@ Start from the maintained [Codex](examples/native/agent-codex.kdl) or
 
 ```sh
 export CATALOG="${XDG_STATE_HOME:-$HOME/.local/state}/st2/default/catalog"
-mkdir -p "$CATALOG/agents/<host>/<identity>" "$CATALOG/_templates"
-cp examples/native/agent-codex.kdl "$CATALOG/agents/<host>/<identity>/agent.kdl"
-${EDITOR:-vi} "$CATALOG/agents/<host>/<identity>/agent.kdl"
+bundle="$(mktemp -d)"
+mkdir -p "$bundle/assets"
+cp examples/native/agent-codex.kdl "$bundle/agent.kdl"
+cp ./composed-AGENTS.md "$bundle/assets/AGENTS.md"
+${EDITOR:-vi} "$bundle/agent.kdl"
+st2 agent publish --catalog "$CATALOG" --bundle "$bundle" --expect-absent --json
 ```
 
-Replace `<host>`, `<identity>`, `<workspace>`, and `<boot prompt>`. Add every file referenced by
-`copy` under `$CATALOG/_templates`.
+Replace `<host>`, `<identity>`, `<workspace>`, and `<boot prompt>`. Include every file referenced by
+`copy` in the bundle. For a later declaration-only update, publish `agent.kdl` with the current
+declaration's SHA-256 via `--spec ... --expect-sha256 HEX`; sibling assets and state are preserved.
 
 The compact declaration shape is:
 
@@ -115,7 +119,7 @@ agent "<identity>" {
   ding
 
   render {
-    copy "_templates/<host>.<identity>.AGENTS.md" "AGENTS.md"
+    copy "assets/AGENTS.md" "AGENTS.md"
     json-upsert ".codex/hooks.json" #"""
 {"hooks":{"SessionStart":[{"hooks":[{"type":"command","command":"$ST_HOOKS/codex-session-start.sh","timeout":5}]}]}}
 """#
@@ -360,7 +364,7 @@ ls, up, down, validate, doctor
 message, ding, agents, status, context, resource
 env, pty, shell, pretrust
 hooks, service, eval
-compile-agent (experimental)
+agent publish
 completions
 ```
 
@@ -420,5 +424,6 @@ interviewer reply at-or-after the exact kickoff receipt completes it. Canonical
 verdict. Without the directive, Agent Spec-shaped files inside a fixture remain inert and compact
 evals retain their flat bus and completion semantics.
 
-`st2 compile-agent` remains experimental. Hand-authored KDL is the canonical st2 authoring
-interface, and generated output must be reviewed before materialization.
+`st2 agent publish --catalog ROOT (--spec FILE | --bundle DIR) (--expect-absent |
+--expect-sha256 HEX)` is the sole catalog-declaration writer. The publisher admits the complete
+prospective catalog under a compare-and-swap lock before making one atomic change.

docs/vrs/requirements.md

@@ -122,3 +122,15 @@ accepted.
   inspection exposes every Resource binding without interpreting its type or URI.
   Resource-only declaration changes do not alter a task's effective launch
   definition and do not stop, replace, or relaunch healthy work.
+- **R22 Transactional catalog authoring:** One st2 publication operation admits
+  exactly one canonical KDL Agent Spec, with explicit host and identity, against
+  the complete prospective catalog. Publication is compare-and-swap, durable,
+  and atomic: readers observe either the previous declaration set or the next
+  complete set. Reconciliation holds one coherent declaration snapshot through
+  materialization, runtime observation, planning, and execution, so a retirement
+  cannot commit and then be followed by a launch from stale catalog input.
+  A durable incomplete-apply marker fences every declaration-plane snapshot and
+  action after a crashed whole-catalog apply; a resident supervisor stays alive
+  but performs zero lifecycle actions until the transaction is completed.
+  Presence, messages, context, and Resource state remain independently writable
+  and are never serialized behind catalog authoring.

docs/vrs/spec.md

@@ -89,6 +89,58 @@ renderer integration in [st2#61](https://github.com/compoundingtech/st2/issues/6
 and the portable Agent Spec envelope in
 [evals#41](https://github.com/compoundingtech/evals/issues/41).
 
+## Transactional catalog authoring
+
+`st2 agent publish --catalog ROOT (--spec FILE | --bundle DIR)
+(--expect-absent | --expect-sha256 HEX) --json` is the sole supported
+declaration writer. It accepts exactly one canonical KDL `agent` node with an
+explicit, path-safe host and identity. st2 no longer exposes an intent compiler:
+external renderers own the transformation from human intent to exact Agent Spec
+bytes or a create-only publication bundle.
+
+The persistent `<catalog>/.st2/catalog-authoring.lock` defines one cooperative
+read/write transaction domain:
+
+```text
+publisher (EX)  : snapshot input -> CAS -> full-catalog admission -> atomic publish + fsync
+reader (SH)     : discover -> materialize/observe -> plan -> execute
+state plane     : message | context | Resource | status                 (unlocked)
+```
+
+The publisher derives the destination from the captured declaration, replaces
+only `agent.kdl` for a hash-authorized update, and preserves all sibling runtime
+state. A bundle is create-only and is renamed from a hidden same-filesystem
+stage; retry reports `unchanged` only when every projected bundle file already
+matches. `--expect-absent` is idempotent for identical input.
+`--expect-sha256` rejects a stale writer. Full-catalog admission rejects any
+structural validation error before publication. The typed result is
+`published` or `unchanged`.
+
+The lock file is never removed: replacing its inode would split the transaction
+domain for processes that already hold it open. Reconciliation holds SH from
+discovery through execution. Validation, doctor, roster, listing,
+materialization-only, targeted reconciliation, and catalog teardown take a
+coherent SH snapshot. State-plane commands deliberately do not: their atomic
+files remain live while a declaration is admitted.
+
+`<catalog>/.st2/catalog-apply-incomplete` is the durable whole-catalog
+transaction fence. Any presence is authoritative, including malformed content.
+The reserved canonical record is:
+
+```json
+{"schema":"st2.catalog-apply-incomplete.v1","desiredStorePath":"/nix/store/..."}
+```
+
+After taking its authoring lock, st2 refuses publication, validation,
+materialization, teardown, roster, doctor, and catalog listing while the marker
+exists. One-shot and selected reconcile fail explicitly. A resident supervisor
+instead remains alive, reports a skipped/incomplete pass, and performs no
+runtime observation or lifecycle action, avoiding a service restart storm.
+Message, context, Resource, and status operations remain available. The future
+whole-catalog apply command owns marker creation, admission, durable apply,
+verification, and clearing inside one transaction; external lock execution and
+bypass flags are not part of the contract.
+
 ## Host-local scheduling and supervision
 
 ```text

examples/README.md

@@ -1,22 +1,23 @@
 # Native st2 examples
 
 [`native/`](native/) contains the maintained, hand-authored Codex and Claude agent declaration
-shapes. Copy one to:
+shapes. Render one outside the catalog as:
 
 ```text
 <catalog>/agents/<host>/<identity>/agent.kdl
 ```
 
-Replace every placeholder, add its referenced files under `<catalog>/_templates/`, then gate the
-result before starting a process:
+Replace every placeholder, put its referenced files in the bundle's `assets/` directory, then publish
+and gate the result before starting a process:
 
 ```sh
 st2 hooks install
 st2 hooks verify
+st2 agent publish --catalog <catalog> --bundle <bundle> --expect-absent --json
 st2 validate --catalog <catalog>
 st2 up --catalog <catalog> --host <host> --materialize-only
 st2 up --catalog <catalog> --host <host> --once
 ```
 
-Hand-authored KDL is canonical. `st2 compile-agent` is an experimental generation aid; inspect all
-of its KDL and workspace targets before use.
+Canonical KDL is the boundary; st2 does not compile intent. Inspect all KDL and workspace targets
+before publication.

examples/native/README.md

@@ -10,22 +10,22 @@ These maintained, hand-authored declarations are the canonical starting points:
 
 The examples use `<host>`, `<identity>`, and `<workspace>` placeholders. st2 provides `CATALOG`,
 `ST_ROOT`, `PTY_ROOT`, and `ST_HOOKS` when it starts a task, so hook declarations contain no
-machine-specific install paths. Copy the appropriate file into
-`<catalog>/agents/<host>/<identity>/agent.kdl`, replace every placeholder, and add the referenced
-catalog-owned templates. `role` is optional metadata; `supervisor` is optional runtime routing.
+machine-specific install paths. Copy the appropriate file outside the catalog, replace every
+placeholder, assemble it and the referenced files under the publication bundle's `assets/`
+directory, then publish the bundle. `role` is optional metadata; `supervisor` is optional runtime routing.
 Uncomment them when the seat has an assigned role or reports to another bus identity.
 
 ## Lifecycle
 
-`compile-agent` is an experimental generation aid. It writes one declaration, catalog-owned
-templates, and the agent's `resources/{inbox,archive,context,links}` directories without changing
-the workspace. Inspect all generated KDL and workspace targets before use.
+st2 accepts exact canonical KDL or a create-only bundle; it does not compile human intent. Inspect
+all KDL and workspace targets before publication.
 
-Use this sequence for hand-authored or generated declarations:
+Use this sequence:
 
 ```sh
 st2 hooks install
 st2 hooks verify
+st2 agent publish --catalog <catalog> --bundle <bundle> --expect-absent --json
 st2 validate <catalog>
 st2 up <catalog> --host <host> --materialize-only
 st2 up <catalog> --host <host> --once

examples/native/agent-claude.kdl

@@ -17,8 +17,8 @@ agent "<identity>" {
   ding
 
   render {
-    copy "_templates/<host>.<identity>.persona.md" ".st2/PERSONA.md"
-    copy "_templates/bus.st2.md" ".st2/bus.md"
+    copy "assets/PERSONA.md" ".st2/PERSONA.md"
+    copy "assets/bus.st2.md" ".st2/bus.md"
     ensure-line ".claude/rules/st2.md" "@../../.st2/PERSONA.md"
     ensure-line ".claude/rules/st2.md" "@../../.st2/bus.md"
     json-upsert ".claude/settings.local.json" #"""

examples/native/agent-codex.kdl

@@ -18,7 +18,7 @@ agent "<identity>" {
   ding
 
   render {
-    copy "_templates/<host>.<identity>.AGENTS.md" "AGENTS.md"
+    copy "assets/AGENTS.md" "AGENTS.md"
     json-upsert ".codex/hooks.json" #"""
 {
   "hooks": {