docs(vrs): define host-local partition contract

Author: myobie

docs/vrs/03-host-local/requirements.md

@@ -0,0 +1,49 @@
+# Host-local supervision — Requirements
+
+Host-local supervision is one machine's application of the root
+[st2 vision](../vision.md). It refines host placement, root supervision, and
+control-plane replacement in
+[R03](../requirements.md#L38-L39),
+[R04](../requirements.md#L43-L46), and
+[R11](../requirements.md#L65-L70). Partition safety remains a fleet-level
+contract in [R18/R22](../requirements.md); this sub-VRS does not redefine it.
+
+## Requirements
+
+- **HOST-R01 One local subject:** A supervisor reconciles one selected catalog
+  for one selected host. Another host's declarations and runtime records are
+  outside that subject.
+- **HOST-R02 Local desired-versus-actual convergence:** The deterministic
+  control plane compares declarations pinned to the selected host with that
+  host's observed task state, adopts matching live work, and starts only
+  genuinely missing work.
+- **HOST-R03 One control-plane writer:** At most one resident control plane may
+  reconcile a given catalog and host at a time. A different host supervising
+  the same synced catalog is a different subject.
+- **HOST-R04 Independent task lifetime:** Stopping, killing, or replacing the
+  control plane does not stop or replace running agent tasks. A successor
+  adopts surviving tasks without duplicating them.
+- **HOST-R05 Explicit destructive lifecycle:** Control-plane absence, restart,
+  or loss of a transport peer is not teardown authority. Local tasks are
+  stopped only by an explicit local lifecycle decision, including a locally
+  applied retirement declaration or teardown command.
+- **HOST-R06 Intelligent local escalation:** The selected host's root agent
+  observes local health, performs bounded recovery, and escalates unresolved
+  failures without turning unavailable peer state into a fleet-health verdict.
+- **HOST-R07 Catalog liveness is not supervisor liveness:** A catalog remains
+  live while any canonical agent belonging to it is running, including while
+  its resident control plane is stopped, restarting, or unavailable.
+  Supervisor state is reported separately. Its absence may delay convergence
+  but does not make continuing agents dead, authorize teardown, or erase the
+  last-applied catalog. DING/sidecar survival alone does not satisfy this
+  agent-liveness predicate. Under incomplete or partitioned observation,
+  absence of evidence cannot prove the catalog globally not live.
+- **HOST-R08 Stable resolved state roots:** While a catalog is live, or while
+  its resident supervisor is running, the resolved catalog root and PTY root
+  remain stable mounted state paths and must not be relocated. Their contents
+  have different semantics; ordinary catalog edit/sync remains allowed.
+  Relocation requires the explicit coordinated migration contract in
+  [issue #85](https://github.com/compoundingtech/st2/issues/85).
+
+Current mechanisms, executable evidence, and unimplemented partition
+activation work are mapped in [spec.md](spec.md).

docs/vrs/03-host-local/spec.md

@@ -0,0 +1,80 @@
+# Host-local supervision — Spec
+
+This is a concise map from the
+[Host-local supervision requirements](requirements.md) to current mechanisms
+and evidence. It does not define transport, remote attachment, deployment
+policy, or a second Doctor health model.
+
+## Subject selection and ownership
+
+The CLI selects a catalog and an explicit or locally detected host. A
+[`HostLock`](../../../src/host_lock.rs#L1-L67) tracks resident ownership of
+that `(catalog, host)` subject independently from another host's slice of the
+same synced catalog.
+
+## Current mechanism and evidence
+
+| Requirement | Current mechanism | Evidence |
+| --- | --- | --- |
+| HOST-R01 | Reconciliation filters every declaration through its resolved host; remote declarations are reported but not acted on. | [`reconcile`](../../../src/reconcile.rs#L106-L191); [host-placement tests](../../../tests/reconcile.rs#L135-L153) |
+| HOST-R02 | One pass discovers declarations, obtains an authoritative local session view, computes desired versus actual, and then adopts or executes the plan. A failed session listing skips the whole pass. | [reconcile pass](../../../src/run.rs#L737-L813); [reconcile-plan model](../../../src/reconcile.rs#L66-L81) |
+| HOST-R03 | The long-running CLI checks for a live owner of the same catalog and host, records its own PID, and reclaims stale ownership. | [catalog supervisor entry](../../../src/main.rs#L1738-L1838); [`HostLock` tests](../../../src/host_lock.rs#L83-L143) |
+| HOST-R04 | PTY and exec tasks survive normal or forced control-plane termination and binary replacement; the successor preserves PID and creation identity while adopting them. | [replacement acceptance](../../../tests/nomad_survival.rs#L592-L701) |
+| HOST-R05 | Normal supervisor exit leaves tasks running. Teardown and retirement are separate paths that target only the selected host's declared task IDs. | [`down` and teardown](../../../src/run.rs#L987-L1056); [explicit-lifecycle acceptance](../../../tests/nomad_survival.rs#L703-L780) |
+| HOST-R06 | The deterministic loop surfaces bounded crash-loop failure to the declared supervisor; the root-agent responsibility itself is owned by root R04. | [crash-loop surfacing](../../../src/run.rs#L1090-L1189); [R04](../requirements.md#L43-L46) |
+| HOST-R07 | Agent/task liveness and the resident supervisor lock are separate observations. A replacement adopts matching work visible in its selected current state. Sidecar-only work does not make an otherwise unrunnable agent live. No global catalog-liveness classifier is implemented. | [replacement adoption](https://github.com/compoundingtech/st2/blob/661c88b6e50cddbdf85e8ffaca9245c46491a1e0/tests/nomad_survival.rs#L608-L717); [separate supervisor report](https://github.com/compoundingtech/st2/blob/661c88b6e50cddbdf85e8ffaca9245c46491a1e0/src/main.rs#L985-L1008); [DING-only boundary](https://github.com/compoundingtech/st2/blob/661c88b6e50cddbdf85e8ffaca9245c46491a1e0/tests/reconcile.rs#L533-L561) |
+| HOST-R08 | Stable catalog-root and PTY-root path lifetime is an accepted constraint. The guided coordinated relocation operation is not implemented. | [migration contract](https://github.com/compoundingtech/st2/issues/85) |
+
+## Catalog liveness and path lifetime
+
+A live canonical catalog agent keeps the catalog live across supervisor
+downtime. The missing supervisor is separate factual state: it can delay
+convergence, but it cannot turn the continuing agent into dead work, erase the
+last-applied catalog, or authorize teardown. On restart, st2 adopts matching
+work that is visible in the selected current state and launches only genuinely
+missing work. A surviving generated DING sidecar without a canonical agent does
+not satisfy the catalog-agent liveness predicate.
+
+This is a host-local contract, not cross-root or global discovery. Under a
+partition or otherwise incomplete observation, st2 cannot infer that a catalog
+is globally not live merely because it cannot currently see a canonical agent.
+Supervisor state remains independently reportable.
+
+While the catalog is live, or while its resident supervisor is running, its
+resolved catalog root and PTY root are stable mounted state paths. Ordinary
+edits and file sync within the catalog remain allowed under the separate
+complete-version and last-known-good rules; this is a path-lifetime constraint,
+not an opaque-database contract. Relocating either resolved root requires the
+explicit coordinated operation in
+[issue #85](https://github.com/compoundingtech/st2/issues/85). Ordinary
+`up`, `doctor`, and reconciliation remain scoped to the currently selected
+paths and never scan arbitrary old roots.
+
+## Partition boundary
+
+The root [R18/R22](../requirements.md) contract requires a complete,
+validated, locally applied catalog to remain authoritative through transport
+loss. The current reconciler instead discovers the live catalog filesystem on
+each pass ([source](../../../src/run.rs#L737-L758)), while validation is a
+separate read-only command
+([source](../../../src/validate.rs#L1-L24)). st2 does not yet stage a candidate
+catalog, assign or compare catalog versions, atomically activate a validated
+snapshot, or retain an activation receipt. That durable last-known-good
+activation path is unimplemented.
+
+Peer reachability is not currently a reconciler input. This is consistent with
+peer absence being neutral, but the declaration shape for an explicit local
+operation that depends on a peer or source is not yet specified.
+
+## Open questions
+
+- What identifies a complete candidate catalog and orders it after the locally
+  applied version?
+- What atomic activation and durable receipt let a replacement control plane
+  recover the last-known-good version after interruption?
+- How does a declaration express a local operation's dependency on a peer or
+  source without turning peer presence into general health?
+- [`HostLock` acquisition](../../../src/host_lock.rs#L26-L48) is currently
+  check-then-write rather than an atomic create. What ownership primitive
+  closes simultaneous first-start races while retaining host-scoped
+  stale-owner recovery?

docs/vrs/requirements.md

@@ -105,6 +105,7 @@ accepted.
 - **R17 Durable error propagation:** Lifecycle, harness/eval, provider-turn,
   task/exec/PTY, hook, and delivery errors are durably reported to the
   responsible supervisor with agent/task identity and actionable context.
+
 - **R19 Targeted reconciliation:** An exact agent/task selector resolves its
   identity and pinned host before mutation; unknown, ambiguous, and wrong-host
   targets refuse before writes, listing, or actions. Materialization, hook
@@ -122,3 +123,15 @@ accepted.
   inspection exposes every Resource binding without interpreting its type or URI.
   Resource-only declaration changes do not alter a task's effective launch
   definition and do not stop, replace, or relaunch healthy work.
+
+### Must remain safe through host partitions
+
+- **R18 Last-known-good local desired state:** During transport loss, each host
+  continues converging locally applied work from its last complete, validated
+  catalog. A missing, partial, or invalid incoming catalog cannot replace that
+  desired state or authorize teardown.
+- **R22 Independent fleet convergence:** Temporary catalog-version skew between
+  hosts is acceptable. On reconnect, a host may apply only a complete,
+  validated, newer catalog and then converges its own work independently. Peer
+  or source reachability is factual state, not a health classification; its
+  absence affects local work only through an explicit local dependency.

docs/vrs/spec.md

@@ -167,6 +167,17 @@ validate ──► materialize ──► host-local st2 scheduler/reconciler
   lifecycle is the explicit authority to resume ordinary replacement.
   `retired #true` remains the separate explicit teardown path.
 
+- **Catalog liveness:** The
+  [Host-local supervision contract](03-host-local/requirements.md) separates
+  continuing canonical-agent liveness from resident-supervisor state. A
+  running canonical agent keeps its catalog live across supervisor downtime;
+  DING/sidecar-only survival does not. Incomplete observation cannot prove a
+  catalog globally not live. While the catalog is live, or its supervisor is
+  running, the resolved catalog root and PTY root remain stable paths; ordinary
+  catalog edit/sync remains allowed, while relocation requires the coordinated
+  operation in
+  [issue #85](https://github.com/compoundingtech/st2/issues/85).
+
 - **Session registry:** A catalog owns the `pty` registry holding its tasks.
   `<catalog>/pty` is the default; a catalog may declare another so that one host
   can share a single registry across catalogs. Resolution is an exported
@@ -179,6 +190,32 @@ validate ──► materialize ──► host-local st2 scheduler/reconciler
   closed instead of hanging reconciliation. The deadline is containment, not
   the mechanism for admitting a larger fleet.
 
+## Partition and catalog activation
+
+- **R18:** Transport loss does not invalidate a host's locally applied desired
+  state. Missing, partial, or invalid incoming catalog state cannot replace the
+  last complete validated version or cause teardown.
+- **R22:** Hosts may temporarily apply different catalog versions and converge
+  independently. Peer or source absence is neutral unless a declared local
+  operation explicitly depends on it; reconnect is not itself evidence that a
+  candidate catalog is complete, valid, or newer.
+
+Current host isolation is source-backed: reconciliation filters declarations
+to one selected host
+([source](../../src/reconcile.rs#L106-L191)), a host-scoped lock detects an
+existing resident writer for the same local subject
+([source](../../src/host_lock.rs#L1-L67)), and running tasks survive control
+plane loss and adoption
+([evidence](../../tests/nomad_survival.rs#L592-L701)).
+
+The durable catalog-activation half of R18/R22 is not implemented. Each pass
+currently discovers the live catalog directory directly
+([source](../../src/run.rs#L737-L758)); `st2 validate` is separate and
+read-only. There is no staged candidate boundary, catalog version ordering,
+atomic validated activation, or durable last-known-good receipt. The exact
+candidate-completeness, version, and activation-recovery contracts remain open
+design work.
+
 ## Message lifecycle
 
 ```text