Skip to content

fix(catalog): admit a prepared plane without runtime workspace dirs - #427

Merged
schickling-assistant merged 1 commit into
mainfrom
schickling-assistant/2026-09-03-2026-09-03-raw-apply-absent-workspace
Sep 3, 2026
Merged

fix(catalog): admit a prepared plane without runtime workspace dirs#427
schickling-assistant merged 1 commit into
mainfrom
schickling-assistant/2026-09-03-2026-09-03-raw-apply-absent-workspace

Conversation

@schickling-assistant

Copy link
Copy Markdown
Contributor

Problem

.workspace is runtime-only: the agent owns its contents, no declaration
describes them, and #425 made the raw preimage capture none of them. The apply
side never learned that. project_excluding still demanded that every declared
workspace fact already exist in the prepared plane, which makes the exact repair
path the raw preimage exists for unreachable — a prepared plane built from a raw
snapshot has no such directory:

$ st2 catalog digest --catalog <live> --prepared <raw-snapshot> --json
Error: inspect prepared workspace fact <prepared>/agents/host/worker/.workspace

Caused by:
    No such file or directory (os error 2)

This is not theoretical. It is currently blocking a dev3 supervisor repair: the
deployer raw-snapshots an invalid live catalog, publishes valid declaration bytes
into that snapshot, and applies against the opaque preimage. On main (fbd1ff6)
that sequence dies at the digest step for any agent whose declared workspace
directory does not happen to exist.

Goal

A prepared plane may omit the directory for a declared workspace fact. Every
other rule holds, and the applied catalog still satisfies the live invariant that
a declared workspace fact is present.

Decisions

  • The fact stays in the projection. Only the pre-existence requirement is
    dropped. materialize_projection already publishes workspace_dirs, so a
    missing one is republished as an empty directory and rootSha256 is unchanged
    — the fact is hashed from the declarations either way.
  • A present directory must still be real and empty. Prepared state may not
    smuggle runtime bytes (or a symlink) through the declaration plane. That
    rejection is untouched and still covered.
  • Uniform across prepared planes, not raw-only. catalog digest --prepared
    has no raw mode, so the deployer's digest-then-apply sequence cannot express
    the distinction at the boundary where it fails. An absent empty directory also
    carries no information the CAS does not already cover, so a raw-only carve-out
    would add a mode without adding a guarantee.
  • Alternative rejected: re-materializing workspace facts in the raw snapshot.
    A raw preimage exists precisely because the declarations may be invalid, so it
    cannot know which directories are workspace facts — that is why fix(catalog): exclude workspaces from raw preimages #425 excluded
    them structurally.

Verification

New regression test reproduces the deployer's shape end to end (raw snapshot of
an invalid live plane → valid declaration bytes written into it → raw apply),
and asserts the fact is republished while live runtime content survives.

Negative control, same test with the source fix reverted:

thread 'raw_preimage_repairs_a_catalog_whose_declared_workspace_fact_is_runtime_only'
  panicked at tests/catalog_apply.rs:1932:5:
Error: inspect prepared workspace fact /tmp/.../agents/host/worker/.workspace

Caused by:
    No such file or directory (os error 2)

With the fix:

$ nix develop -c cargo test --test catalog_apply raw_preimage
running 8 tests
test raw_preimage_rejects_hard_linked_declarations ... ok
test raw_preimage_treats_the_live_envelope_and_pty_root_as_bytes ... ok
test raw_preimage_repairs_a_catalog_whose_declared_workspace_fact_is_runtime_only ... ok
test raw_preimage_migrates_legacy_argv_profile_to_a_component_catalog ... ok
test raw_preimage_repairs_an_invalid_catalog_and_preserves_mutable_state ... ok
test raw_preimage_accepts_valid_bytes_and_wrong_cas_preserves_declarations ... ok
test raw_preimage_repairs_catalogs_with_unadmitted_profile_modules ... ok
test raw_preimage_resume_uses_the_durable_validated_stage ... ok
test result: ok. 8 passed; 0 failed; 52 filtered out

Whole-file comparison against pristine main at fbd1ff6, same machine, same
command (nix develop -c cargo test --test catalog_apply):

main (fbd1ff6)   45 passed; 14 failed
this branch      46 passed; 14 failed

The 14 failures are identical in name on both sides — this change adds no
failure and fixes none of them. See Concerns.

Complexity

No new abstraction. One admission predicate goes from "must exist" to "must be a
real empty directory if it exists", plus the spec sentence that stated the old
rule.

Concerns

  • catalog_apply has 14 pre-existing failures on main, unchanged here. Two
    shapes: several time out waiting for a helper's ready file (this machine was
    under load average ~90), and
    workspace_facts_are_empty_in_prepared_admitted_against_live_and_never_applied
    fails at its final apply with
    [root-count]: host 'host' must declare exactly one root agent; found 2 — its
    fixture declares two unsupervised agents. Reproduced with this branch's changes
    reverted, so it is not from this PR, and repairing that fixture is out of scope
    here. Worth a follow-up: it hides real breakage in exactly this file.
  • That same test is edited here, minimally: its "missing prepared workspace fact
    is rejected" case asserted the contract this PR removes, so it is gone. The
    "nonempty is rejected" case stays.

Friction & bottlenecks

  • The failure surfaces as inspect prepared workspace fact — an I/O context
    string on a NotFound — which reads like a permissions or path problem rather
    than "this admission rule is unsatisfiable for raw preimages". A rule that
    cannot be satisfied by a supported input shape deserves its own message.
  • The pre-existing red suite meant the first signal of this bug came from a
    downstream Nix check rather than from st2's own tests.

Follow-ups

  • Repair the workspace_facts_... fixture (two root agents) and get
    catalog_apply green so regressions in it are visible.

References

Refs #425, #424

Posted on behalf of @schickling
field value
agent_identity dev3.direct.omp.quzh99p3
session dev3.quzh99p3
agent_persona generalist
agent_supervisor unavailable
agent_tool OMP
agent_tool_version 18.1.2
agent_runtime OMP 18.1.2
tooling_profile dotfiles@9323c3a

`.workspace` is runtime-only: the agent owns its contents, nothing declares
them, and #425 made the raw preimage capture none of them. The apply side kept
requiring every declared workspace fact to already exist in the prepared plane,
so the exact repair path the raw preimage exists for became unreachable: a
prepared plane built from a raw snapshot has no such directory, and
`digest --prepared` failed with

    inspect prepared workspace fact <prepared>/agents/<host>/<id>/.workspace
    Caused by: No such file or directory (os error 2)

Admission now tolerates an absent directory and keeps every other rule: a
directory that IS there must still be a real, empty one, so prepared state
cannot smuggle runtime bytes through the declaration plane. The fact itself
stays in the projection, so `materialize_projection` republishes it as an empty
directory and the applied catalog still satisfies the live invariant that a
declared workspace fact is present.

The relaxation is uniform across prepared planes rather than raw-only:
`catalog digest --prepared` has no raw mode, so the deployer's
digest-then-apply sequence cannot express the distinction, and an absent empty
directory carries no information the CAS does not already cover — the declared
fact is hashed either way.

Regression test covers the deployer's shape end to end: raw-snapshot an invalid
live plane whose declared workspace is runtime-only, publish valid declaration
bytes into that snapshot, and bind the apply to the opaque preimage. Without
this fix it reproduces the error above verbatim.

The existing workspace-fact test loses its "missing" case, which asserted the
contract this fixes. Its remaining "nonempty" rejection is unchanged.

agent-identity: dev3.direct.omp.quzh99p3
agent-persona: generalist
agent-supervisor: unavailable
agent-tool: OMP
agent-tool-version: 18.1.2
agent-runtime: OMP 18.1.2
tooling-profile: dotfiles@9323c3a
@schickling-assistant
schickling-assistant marked this pull request as ready for review September 3, 2026 12:56
@schickling-assistant
schickling-assistant merged commit 7cb7f64 into main Sep 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant