Skip to content

Commit d72ddde

Browse files
fix(resync): publish off the reconcile pass's thread
A reconcile pass could not complete while resync publication was making no progress. `install_live` and `deactivate` block on an acknowledgement from the resync worker thread, and that same thread ran every publication — which takes the shared catalog-authoring lock, re-resolves the catalog, and takes the recipient's stream lock. So a publication that was slow, refused and retried, or blocked on another process serialized the whole pass behind it, once per live seat. That is the property that let a terminal-refusal loop deny reconcile passes for two hours: the loop's only power was denying the pass that would have ended it. Publication now runs on its own thread. The worker captures a transition, hands it off, and applies the outcome when it returns, so it stays the only writer of carrier baselines and retry deadlines. One publication is outstanding per subscription at a time; a queued publication is dropped when its subscription is deactivated or removed by a refresh, so nothing is published to a seat the pass has already decided receives no events. Re-observing the carrier after an outcome is the single re-arm mechanism for both a retained refusal and a carrier that moved on while the publication was outstanding. This is the primary requirement from #431. Terminal-refusal classification is separate and still wanted: it removes the CPU burn and the unbounded retry, but it would leave this coupling intact for the next condition that starves pass completion. Refs #431 agent-identity: dev3.compoundingtech.st2.resync-lock.worker agent-persona: worker agent-supervisor: dev3.compoundingtech-lead agent-tool: OMP agent-tool-version: 18.1.2 agent-runtime: OMP 18.1.2 tooling-profile: dotfiles@7534055
1 parent 5b2ee7c commit d72ddde

3 files changed

Lines changed: 544 additions & 95 deletions

File tree

INVARIANTS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,4 +36,5 @@ materialization, messaging, DING, or presence must preserve them.
3636
| **Native flat root** | Without an authored override, catalog tasks, eval messaging, shell helpers, and DING all use the catalog itself as `ST_ROOT`; no nested bus directory is synthesized. | `src/eval_run.rs::bus_root_expands_st_root_else_defaults`; `tests/eval_run_e2e.rs::st2_eval_runs_a_benign_folder_to_a_pass_verdict`; `tests/pty.rs` |
3737
| **Resource observation is state-first, atomic, and fenced** | ABI-3 periodic publication and demanded `Published` results reuse one bounded `Publication` payload and one host acceptance, digest, relevance, typed-fact, and catch-up core; the host never trusts a runtime digest or observation timestamp. Demand reaches only a resident runtime that explicitly declares `capability "demand"`. Every `Observe` carries a positive watermark and the exact owner, binding, and registration, and exactly one matching `Unchanged`, `Failed`, or `Published` atomic result closes it. One outstanding dispatch plus one latest trailing watermark coalesces bursts without losing in-flight arrivals. Backpressure retains queued demand, replacement fences stale output, restart and provider failure settle honestly, and client disconnect or wait expiry never cancels accepted work. | `tests/resource_profile_supervisor_e2e.rs::demand_observation_settlement_matrix_is_atomic_and_preserves_facts`; `tests/resource_profile_supervisor_e2e.rs::demand_observation_coalesces_and_fences_watermarks`; `tests/resource_profile_supervisor_e2e.rs::demand_observation_survives_restart_disconnect_and_denies_missing_capability`; `tests/resource_profile_supervisor_e2e.rs::observable_publication_reaches_builtin_resync_with_filter_catch_up_and_scope_isolation`; `tests/agent_resource.rs::refresh_cli_reports_exact_receipts_and_wait_expiry_keeps_the_request`; `src/resource_observe.rs::tests::receipt_evidence_shape_matches_atomic_results` |
3838
| **Atomic resource proposal publication** | Every changed resource publication is one host-owned compare-and-swap fenced by binding generation, state revision, and prior carrier digest. A persistent cross-process lock admits at most one proposal from the same prior. The content-derived proposal ID binds the accepted carrier digest and semantic outbox envelope; the durable intent becomes eligible only with the exact canonical carrier, then folds into one authoritative catch-up state. A pre-carrier crash exposes old state, a post-carrier crash catches up on restart, and retry after a lost acknowledgement returns the durable receipt without another transition. Ordinary reconciliation fails closed on out-of-band divergence after a committed intent; only an explicit generation-advance recovery may re-adopt the canonical carrier or its absence while invalidating the old intent and fence. | `src/resource_profile.rs::tests::atomic_publication_fences_races_and_survives_crash_restarts`; `src/resource_profile.rs::tests::generation_advance_explicitly_recovers_diverged_or_missing_carrier` |
39+
| **Pass progress is independent of publication** | A reconcile pass completes whether or not resync publication is making progress. Publication runs on its own thread, so a pass's per-seat watch-set handshakes never wait for a catalog lock, a stream lock, or a refused publication. One publication is outstanding per subscription at a time, and a queued publication is dropped when its subscription is deactivated or removed by a refresh, so nothing is published to a seat the pass has already decided receives no events. | `src/run.rs::reconcile_pass_completes_while_a_resync_publication_is_blocked`; `src/resync.rs::a_flush_never_hands_off_a_subscription_whose_publication_is_outstanding`; `src/resync.rs::deactivation_drops_only_that_recipients_queued_publication`; `src/resync.rs::a_refresh_drops_a_queued_publication_for_a_subscription_it_removed` |
3940
| **Proof references resolve** | Every qualified test named in this table exists in its named source file, so stale invariant claims fail the suite instead of silently surviving a refactor. | `tests/invariants.rs::qualified_proof_references_resolve` |

0 commit comments

Comments
 (0)