Define renderer-owned harness sessions and the generic st2 lifecycle boundary

[schickling-assistant]

Outcome

This issue is the umbrella for renderer-owned harness session identity and the generic st2 lifecycle boundary.

The ownership decision is:

• Axe allocates the stable logical Agent Session ID.
• Harness-specific drivers understand Codex, Claude Code, native session IDs, transcript layouts, compatibility, and provider flags.
• Axe/the renderer compiles that knowledge into one complete ordinary Agent Spec launch contract: exact argv, environment, rendered files, and optional opaque correlation metadata.
• Core Agent Spec and the st2 up daemon remain harness-agnostic. They do not parse provider transcripts, synthesize provider flags, or choose between provider-specific fresh/resume behavior.
• st2 owns generic task observation, launch-generation comparison, fencing, recreation, receipts, and reconciliation.

The normalized catalog is therefore a compiled boundary, not the place where Codex or Claude lifecycle wisdom is implemented.

Compilation boundary

Harness-aware authoring may use driver-specific input such as the following non-normative shape:

claude {
  session {
    id "arn:agent:<scope>:session:<id>"
    native-id "<provider-native-id>"
    restore "resume-and-rewake"
  }
}

That input is consumed before core st2 reconciliation. The renderer emits one complete launch contract, conceptually:

agent "cos" {
  argv "axe" "agent" "launch" "<fully rendered axes>"
}

The exact authoring syntax remains open. The invariant is that the daemon sees a complete command rather than partially interpreting harness session policy.

Identity boundaries

The system must distinguish:

1. stable Agent Spec identity;
2. Axe-allocated logical Agent Session ID;
3. provider-native session ID resolved by a harness driver;
4. st2 Agent Run ID for one successful launch occurrence;
5. backend runtime incarnation/generation;
6. desired and observed launch-contract generations; and
7. the host currently authorized to write a portable session store.

None may be silently inferred from a PTY ID, PID, cwd-based last-session lookup, display label, or another identity in this list.

Responsibilities

Axe and harness drivers

• allocate the logical Agent Session ID;
• resolve or create the exact provider-native session;
• distinguish passive restore, resume-and-rewake, fresh start, and any provider-specific fallback;
• validate transcript/store compatibility and uniqueness;
• render the complete command, environment, and required files; and
• expose enough observed lineage for a launch receipt to prove which native session was selected.

st2

• fingerprint spawn-affecting inputs in the compiled task contract;
• report desired-versus-observed launch generation without automatically restarting healthy work (Expose desired versus observed launch generation for adopted tasks #40);
• mint and expose one Agent Run ID per successful launch (Model Agent Run identity in task inventory #145);
• adopt live tasks without interpreting harness state;
• perform one exact generation-fenced recreation from a compiled declaration (Add an exact generation-fenced recreate operation for one live task #110); and
• keep retirement and newer desired-state authority dominant across lifecycle races.

Fabric or another store transport

Portable session storage and cross-host movement require a separate single-writer ownership protocol. File convergence alone does not authorize a second writer. A move must fence the source, prove a resumable store generation, converge it, transfer authority, and only then launch the compiled destination contract.

Remaining design work

• Ratify Define Axe harness drivers and extract all harness-specific behavior from st2 core #162's Axe driver interface and renderer-owned harness blocks without moving provider semantics into st2 core.
• Ratify logical Agent Session ID format, scope, persistence, and allocation failure behavior.
• Define how opaque logical and native correlation evidence is supplied to inventory without teaching st2 provider semantics.
• Define provider-specific receipts for passive restore, resume-and-rewake, fresh start, and explicit fallback.
• Define the portable store bundle, compatibility version, sensitivity/retention policy, and size limits.
• Define the cross-host single-writer transfer protocol and its failure recovery.
• Add paired Codex and Claude controls for exact resume, fresh creation, ambiguity refusal, missing/incompatible state, and concurrent first launch.

Child work

• Define Axe harness drivers and extract all harness-specific behavior from st2 core #162 - Axe harness drivers, including provider launch/session behavior, hooks, DING evidence, configuration, validation, and session presentation
• Expose desired versus observed launch generation for adopted tasks #40 - desired versus observed launch generation
• Add an exact generation-fenced recreate operation for one live task #110 - exact generation-fenced recreation from the current compiled declaration
• Model Agent Run identity in task inventory #145 - Agent Run identity and supplied session correlation in task inventory
• Activation: deploy PTY environment persistence for manual restart #30 - activation proof for managed PTY environment replay
• Define a guided migration operation for state-bearing st2 paths #85 - guided movement of state-bearing st2 paths

PR #157 explored core start/resume method selection. Its evidence remains useful, but that API is superseded by this renderer-first ownership boundary.

[myobie]

@myobie look at this

[schickling-assistant]

Measured provider behaviour for Codex and Claude Code

This issue asks for "provider-specific behavior tested for at least Codex and Claude Code". We ran that matrix in an isolated transcript store (never the shared one), with an independent adversarial re-derivation afterwards that tried to refute each claim rather than confirm it. Provider versions: Claude Code 2.1.220, Codex 0.145.0. Everything below is version-scoped; none of it should be treated as a stable provider guarantee.

1. Caller-declared native session ID is asymmetric across providers

	Claude Code 2.1.220	Codex 0.145.0
caller declares the native ID at create	yes — --session-id <uuid> is honoured	no
resume preserves the native ID	yes	yes

For Claude, a declared UUID is genuinely minted and persisted: the transcript lands at the path derived from that exact UUID and every persisted record's sessionId equals it. Both were checked, because a file merely named after the requested UUID would look identical to success. Reusing an already-live ID is rejected with "already in use".

For Codex, --session-id exits 2 as an unexpected argument at top level and under exec, resume, and fork; the positional SESSION_ID selects an existing source only. We also probed the config-key loophole: -c session_id=<uuid> is accepted by the config parser and then ignored, minting a different thread ID. A nonsense control key is accepted the same way, so config-parser acceptance is not evidence of capability.

Consequence for the boundary in this issue: this asymmetry is a good argument for the stated design boundary rather than against it. A provider-neutral fresh/resume lifecycle against a logical store absorbs it entirely — one renderer declares the native ID up front, the other must record an ID the provider minted. If the normalized Agent Spec carried a native session ID, that field would be authorable for one provider and only ever observed for the other.

2. Resume does not have to submit a turn

Both providers, resumed without a trailing prompt argument, restore the conversation and wait at the prompt. No conversation turn is added.

This matters for method "resume": a renderer that appends a prompt after the resume target turns "continue this session" into "continue this session and act now". Ours did, which is how we found it. A resume method should be able to restore state without advancing it.

3. "No bytes written" and "no conversation turn" are different guarantees

This is the claim our verifier narrowed most, and it bears on any receipt or store-generation design here:

• Claude: promptless resume added no user/assistant record, but termination appended mode and then permission-mode metadata records — file mtime landed 316 ms before PTY exit, pinning the write to shutdown. Ordinary resume-with-prompt likewise appended one real turn plus a mode record. So the transcript file changes even when the conversation does not.
• Codex: no post-restore write was evidenced at all; source mtime stayed earlier than the resuming process's start.

We could not prove "byte-stable while parked" for Claude — no mid-park hash was captured, so that specific claim is unproven, not confirmed. If a receipt is meant to attest that a store is quiescent, it should key on parsed conversation records rather than file bytes or mtime, at least for Claude.

4. Fork preserves the source but copies history semantically

Relevant to "How are fork, compaction, retention, archive, and deletion represented?":

• Both providers mint a new ID on fork and leave the source byte-identical (verified by hash before/after).
• Codex records forked_from_id in the target — explicit lineage.
• Claude remaps record UUIDs and sessionId in the fork. The history is carried semantically, not as a byte-identical record prefix.

So fork is not an identity-preserving continuation; it is a copy with a new identity. It is, however, the only primitive here that leaves the source provably untouched by construction rather than by care — worth distinguishing from resume in the method vocabulary.

Method notes

Isolated provider home and transcript store throughout; fresh UUIDs only; no session belonging to a live agent was resumed or forked. The independent verifier re-ran minimal cases where the original artifact set could not settle a claim, and reported two of the original safety assertions as honestly unproven for lack of a pre-run baseline. We are reporting the confirmations and the gaps together, since the gaps are the part a later reader would otherwise have to rediscover.

Happy to run further cases against a specific proposed shape — in particular the "simultaneous first launch for one logical store produces at most one authoritative native ID" claim from the evaluation direction above, which the Claude "already in use" rejection suggests may be testable directly.

Posted on behalf of @schickling

field	value
agent_name	unknown
agent_tool	Claude Code
agent_tool_version	2.1.220
agent_runtime	Claude Code 2.1.220
agent_session_lookup	unavailable
tooling_profile	dotfiles@unknown-dirty

[schickling]

Live-fleet cutover evidence: resume identity held, but lifecycle intent is still overloaded

We have now exercised this against a live st2 fleet rather than an isolated store: 10 active Claude Code 2.1.220 sessions were moved between local account/config profiles under st2 0.1.0+0304aa6, each with one exact provider-native session ID. One additional target was concurrently retired by its supervisor and was deliberately left absent.

The good result: all 10 active targets continued the intended native session. For each target we verified all of the following, rather than accepting argv intent alone:

• the launch receipt lowered the typed lifecycle intent to --resume <exact-id>;
• the selected account/config environment was the destination profile;
• the runtime resume tag named the same exact ID and transcript path;
• the transcript contained exactly that one sessionId; and
• post-cutover events appended to that same transcript rather than a newly minted neighbor.

A few design lessons fell out of the operation.

1. Passive restore and resume-and-rewake are different lifecycle methods

Our adapter currently lowers resume to:

claude --resume <id> "You resumed ... reconcile ... then continue"

That preserves native identity, but it also submits a new prompt. In addition, Claude runs the managed SessionStart hook again, which drains externalized context and inbox state. To an attached operator this looked like a cold start even though the transcript proved continuity.

So this was a successful identity-preserving resume, but not a passive/promptless restore. The prior isolated result in this issue already showed that promptless resume is possible. The live run makes the UX and contract consequence concrete: the lifecycle vocabulary should distinguish at least:

• restore exact native state and park without adding a turn;
• resume exact native state and explicitly re-wake/reconcile; and
• create fresh native state with the ordinary boot ritual.

A generic resume bit plus an adapter-chosen trailing prompt hides a consequential policy decision.

2. A launch receipt needs observed lineage, not only declared argv

The strongest proof was not the Agent Spec, PTY name, PID, or even the rendered --resume argument. It was the joined tuple:

spec intent
  -> canonical launch receipt
  -> selected account/config environment
  -> resolved transcript path + admission digest
  -> observed single native session ID
  -> post-launch append to that same lineage

PTY identity stayed stable while process incarnations changed, which is desirable, but it means PTY identity alone cannot attest session continuity. A first-class lifecycle receipt should make this joined proof queryable without reconstructing it from provider files.

Also, the transcript digest captured at admission is a source-generation receipt, not a promise of byte stability: Claude writes metadata and new events during resume. It should be named/treated accordingly.

3. Catalog publication and disruptive lifecycle action are separate phases

Publishing the new account/resume intent was non-disruptive: the supervisor adopted the already-running process. The actual cutover required an explicit per-seat lifecycle boundary, after which the supervisor reconciled the declaration.

That separation is useful, but today the operator action is effectively “kill this managed PTY and wait for reconciliation.” A first-class operation such as “recreate this task using the currently declared lifecycle method” could:

• re-read the declaration under CAS;
• record the before incarnation and desired lifecycle generation;
• perform the boundary;
• wait for the matching observed receipt; and
• return one success/refusal result.

This would also avoid confusing “new PID exists” with “the requested transcript and account were actually selected.”

4. Re-check desired state across the boundary

One migrated verifier received a correct resume receipt and was then marked retired #true by its supervisor concurrently. st2 correctly tore it down. We preserved that newer authority rather than reviving it.

A lifecycle operation therefore needs a final desired-state/CAS check, not just a preflight check. Retirement or another declaration change racing the cutover must win or fail the operation explicitly.

5. Account binding and session identity were orthogonal here, but portability was not proved

The cross-account cutover worked because both local account profiles intentionally referenced the same transcript store. That proves an exact native session can be resumed under a different authenticated profile when the store is shared. It does not prove the catalog-relative portable-store or cross-machine fencing design proposed above; credentials stayed outside the transcript store and no ownership transfer between hosts occurred.

6. Fleet selection should be evidence-based and bounded

We selected targets from the exact live quota message, then resolved each one to a provider-native ID using start-time correlation plus transcript content. We did not migrate every session merely because it used the depleted account. A late sweep found one additional newly blocked session, which was migrated in the same bounded way.

This suggests a useful separation between:

• account eligibility/placement policy;
• declared lifecycle intent;
• native lineage resolution; and
• the operator-selected set of task incarnations to recreate.

The live outcome was good—10/10 active target lineages continued—but the run strongly supports making passive restore, resume-and-rewake, and fresh boot explicit rather than leaving that distinction inside renderer argv.