Skip to content

Latest commit

 

History

History
435 lines (332 loc) · 18.9 KB

e2e-contract.md

File metadata and controls

435 lines (332 loc) · 18.9 KB

Coven Threads end-to-end verification contract

Status: P0 implementation contract
Risk class: R4 — identity/authorization/persistence boundary
Canonical compatibility pin: e2e/compatibility.toml

1. Why this layer exists

coven-threads-core is intentionally a side-effect-free validator library. Its unit and conformance suites prove typed gate, audit-schema, portability, and fail-closed semantics in process. They do not prove that a real client request crosses the Coven daemon boundary, invokes the current Threads checkout, and produces the expected filesystem, SQLite, pending-proposal, scheduler, and restart outcomes.

The missing proof is therefore a boundary-spanning journey:

current coven-threads change
    -> real coven daemon process
    -> real supported request boundary
    -> coven_threads_core gate from this checkout
    -> ephemeral familiar workspace
    -> ephemeral coven.sqlite3 + pending state
    -> scheduler/replay/restart
    -> optional real Cave acceptance journey

This contract does not turn Threads into a daemon or UI repository. It defines what downstream implementations must prove before a Threads change is treated as compatible.

2. Canonical ownership

Concern Canonical owner E2E responsibility
Familiar identity and protected invariants familiar-contract Supply pinned normative fixtures; Threads must not redefine identity.
Protected mutation verdict and audit schema coven-threads Define typed inputs, verdicts, reason codes, audit constraints, and vectors.
Principal authentication, storage, staging, scheduling, apply, recovery coven Launch the process and own every effect.
Orchestration context and attempt lifecycle psyche Consume authority evidence; never manufacture a verdict.
Human rendering and approval forwarding coven-cave Render daemon truth; never become an approval authority.

There is one audit store: ward_audit inside the daemon-owned coven.sqlite3. The harness must not create a second audit truth.

3. Required topology

The required daemon suite lives in OpenCoven/coven, because that repository owns the executable, transport, persistence, staging, and scheduler. A Threads PR checks out both repositories and overlays the current Threads crate into the pinned Coven checkout:

[patch."https://github.com/OpenCoven/coven-threads"]
coven-threads-core = { path = "../coven-threads/crates/coven-threads-core" }

Before executing any journey, the job must prove the override is active through cargo metadata or cargo tree. Testing Coven's historical Git revision while claiming to test the pull request is a hard failure.

Two downstream references are maintained:

  1. Required compatibility pin — a reviewed full commit SHA from e2e/compatibility.toml. This is the stable pull-request gate.
  2. Latest-main canary — a scheduled non-blocking run against coven/main. This detects ecosystem drift without making unrelated downstream movement nondeterministically break Threads pull requests.

4. Isolation and safety

Every run must allocate unique, disposable state:

  • temporary COVEN_HOME;
  • temporary familiar workspace;
  • temporary daemon socket/port;
  • temporary SQLite database;
  • temporary pending/staging directory;
  • synthetic principal, familiar, Ward, and content fixtures;
  • deterministic run identifier included in every artifact.

The suite must never:

  • read or modify a developer's real ~/.coven;
  • use production credentials or real familiar declarations;
  • call gate_protected_edits directly in place of the daemon transport;
  • bypass authentication, request adoption, staging, scheduler, or replay code;
  • infer success solely from an HTTP status while ignoring disk and audit state;
  • sleep on wall-clock time to establish correctness;
  • upload secrets, private prompts, or personal identity content as artifacts.

Unknown state, unavailable evidence, an inactive override, or an ambiguous fixture must fail the suite closed.

5. Required black-box journeys

The first required suite contains eight journeys. Each journey asserts the public response and all applicable persisted effects.

J1 — bounded authorized write

A correctly authenticated and bound principal submits a permitted write.

Required assertions:

  • daemon returns the typed permit/applied disposition;
  • the intended file changes exactly once and atomically;
  • an authoritative validation row is appended;
  • any applied-write row carries the expected previous/next hashes and byte count;
  • no pending proposal remains.

Until the signed principal-authorization profile lands, the fixture must use the strongest current daemon-owned authorization path and state that limitation in its evidence. A client-supplied identity string is never sufficient proof.

J2 — unsigned or unbound protected write

An unbound requester attempts a protected mutation.

Required assertions:

  • request is rejected;
  • protected bytes are unchanged;
  • no proposal or approval route can later apply the write;
  • one typed rejection audit row exists;
  • protected content is not echoed in the error.

J3 — out-of-band drift

After establishing a baseline, mutate a governed surface outside the daemon and submit a conflicting edit.

Required assertions:

  • the drift is detected from materialized bytes;
  • the protected surface is not overwritten;
  • any degrade-to-proposal result is explicitly non-executed;
  • pending artifact, response, and audit row bind the same proposal/diff digest.

J4 — identity invariant mutation and replay

Submit a candidate that violates a declared identity predicate, then exercise both intake and delayed replay.

Required assertions:

  • intake fails closed or stages only where the governing contract permits;
  • deadline replay evaluates the same authoritative predicate set;
  • changed or unavailable evidence cannot apply;
  • descriptor text is never treated as authority.

This journey is the closure evidence for threads-okc.

J5 — protected SOUL.md route prohibition

Attempt every known proposal/stage/approve route for a Tier-0 protected SOUL.md change.

Required assertions:

  • the proposal pipeline cannot apply the protected change;
  • a principal-authorized protected update, where supported, uses a distinct audited authority path;
  • supplying a fingerprint or approval identifier cannot convert the proposal route into protected-write authority.

This journey is the closure evidence for threads-dgg.

J6 — complete veto-window terminal matrix

Open a real delayed-apply window and force every terminal family:

  • applied;
  • vetoed;
  • evidence diverged;
  • revalidation failed;
  • superseded.

Required assertions:

  • every opened window receives exactly one terminal event;
  • every terminal event contains the normative typed close reason;
  • no duplicate terminal event can be appended;
  • a human approval path without a veto window remains valid;
  • an inconsistent human-path plus opened-window history is rejected, not normalized by dropping close detail.

This journey is the closure evidence for threads-980 and the review gate for PR #27.

J7 — retired-Ward schedulability

Pass a valid repository-owned retired-Ward corpus case through the live daemon.

Required assertions:

  • daemon classifies and persists the proposal;
  • scheduler makes the pending interval observable for min_visible;
  • restart recovery re-adopts the pending work;
  • the proposal reaches one valid terminal decision without loss or double apply;
  • unsupported corpus input fails closed.

This journey is the closure evidence for threads-zav.

J8 — committed-evidence drift and restart

Open a delayed window, stop the daemon, alter committed evidence or authority, and restart.

Required assertions:

  • replay detects the mismatch;
  • stale policy, revoked approval, changed familiar binding, changed runtime, or changed proposal bytes cannot be reused;
  • restart neither loses nor duplicates the terminal disposition;
  • file mutation and terminal audit evidence agree after recovery.

6. Deterministic time

Deadline behavior must use an injected clock or scheduler seam. Real sleeps are not acceptable correctness evidence. The test interface must support:

  • advancing to just before min_visible;
  • advancing to the exact earliest close;
  • advancing beyond deadline;
  • restart with persisted logical time inputs;
  • revocation or evidence change between check and commit.

If the production scheduler cannot currently accept deterministic time, the clock seam is a production prerequisite—not permission to weaken assertions.

7. Atomicity and TOCTOU proof

For any journey that applies bytes, final Gate-4 revalidation and the committed binding must refer to the same immutable snapshot or transaction boundary. Evidence must include:

  • request/proposal digest;
  • exact Threads and Coven commits;
  • familiar/root/revision or current equivalent binding;
  • policy/Ward/weave digest;
  • principal authorization reference;
  • pre-write and post-write content digests;
  • audit terminal event identifier;
  • committed file-state digest.

A relevant change after decision but before commit must refuse dispatch or commit. The daemon must not reconstruct authority from incomplete client fields.

8. Failure evidence bundle

On failure, write a sanitized bundle beneath the configured artifact root. The local default is target/e2e-artifacts/<run-id>/. CI must use an uncached root and an upload name scoped to the workflow run and attempt, so restored build caches or repeated attempts cannot supply another run's evidence. The bundle contains:

manifest.json              exact commits, command, platform, scenario
request.json               synthetic redacted request
response.json              daemon response
logs/daemon.log            sanitized process log
state/ward-audit.jsonl      ordered relevant rows
state/pending-tree.txt      pending artifact inventory
state/workspace-tree.txt    governed workspace inventory and hashes
state/sqlite-schema.txt     relevant schema fingerprint, never real user data
junit.xml                  machine-readable result

manifest.json must record whether the local Cargo override was proven active. Artifacts use synthetic data only and default to the retention in e2e/compatibility.toml.

The Coven integration uses the test-only COVEN_THREADS_E2E_ARTIFACT_ROOT override for CI isolation. Older artifacts with commit-only names must be retrieved by exact artifact ID. These storage conventions do not change the required compatibility pin or turn a diagnostic retry into first-attempt acceptance.

9. Red -> fix -> green closure rule

A Phase-5 blocker is not closed by prose or another in-process fixture alone. Its implementation pull request must contain:

  1. a failing black-box test against the pre-fix behavior;
  2. the production fix in the canonical owning repository;
  3. a passing black-box test after the fix;
  4. a lower-level regression test for fast diagnosis;
  5. the named CI check and failure artifact;
  6. migration and rollback notes where persisted state changes.

Human sign-off gates remain human. Under the adopted solo-maintainer policy, Val owns coherence acceptance and the subsequent scoped freeze/reaffirmation. Agents may assemble evidence and recommend a decision; they must never simulate either human decision or claim independent human approval.

10. CI rollout

Observation

  • keep cargo test --locked --workspace as the compatibility baseline;
  • add Nextest with JUnit and fail-on-flaky semantics;
  • collect coverage without enforcing a threshold;
  • measure discovered-test parity and runtime.

Boundary

  • land the daemon fixture downstream;
  • run the eight journeys as non-required checks;
  • verify diagnostic bundles and deliberate red cases.

The advisory workflow .github/workflows/daemon-observation.yml provides a scheduled latest-main canary and a manual candidate observation. Manual runs require a full Coven commit SHA; scheduled runs resolve main and record its exact SHA. Neither adds a required PR check or replaces the stable pin. The workflow becomes schedulable only after landing on the default branch.

scripts/run-daemon-canary.mjs requires clean, disposable checkouts and a new artifact directory. It installs a downstream Cargo configuration overlay so the harness's nested metadata queries see the same patch, preserving existing downstream network/build settings. Ambiguous config filenames and conflicting Cargo patch tables fail rather than replacing an existing configuration. The runner explicitly resolves the overlay lockfile, then runs the feature-enabled threads_e2e target with --locked. Metadata must identify the exact current Threads manifest on coven-cli's resolved dependency edge, not merely another local Threads copy. The runner records original/overlay config and lockfile hashes, exact revisions, command, run/attempt, and the preflight/override/daemon outcome in observation.json. The disposable Coven checkout retains its configuration and lock overlay for diagnosis. It must not be a developer's active worktree.

The recorded configuration overlay must remain the sole regular Cargo config file after each metadata query and daemon execution. Changed or missing bytes, symlink replacements, and newly ambiguous config filenames fail the observation, even if the dependency graph still selects the current Threads checkout. Both checkout HEADs and source cleanliness are rechecked after every Cargo metadata query and daemon execution. Only the exact recorded Coven config overlay and Cargo.lock are exempt from the source check; their bytes are checked separately. Staged, unstaged, and non-ignored untracked source changes fail rather than attributing the result to the original clean revisions. The resolved overlay lockfile is also rechecked after daemon execution and the final metadata query before a passing receipt is written. These boundary checks do not attest to transient changes restored between commands, ignored build outputs, or hermetic execution.

Until the selected downstream revision has the full harness, observation fails explicitly rather than skipping or substituting a library test. A failed advisory workflow remains visible but is not a protected-branch requirement. Its run-scoped evidence is uncached; retries cannot reuse an existing artifact directory or replace first-attempt history.

The downstream observation compiler is pinned separately to Rust 1.95.0: the inspected daemon's locked sysinfo 0.39.6 requires Rust 1.95. Threads' own toolchain, MSRV, and required compatibility baseline remain 1.88.0. Observation records the actual Cargo and rustc versions; a newer daemon requirement must fail visibly until this separate pin is deliberately updated.

Enforcement

  • require the pinned Linux daemon suite after all eight journeys are stable;
  • protect main with quality, Cargo compatibility, and daemon E2E checks;
  • fail if the Cargo override is inactive.

.github/workflows/daemon-compatibility.yml supplies the separate Pinned daemon compatibility job for every pull request and main push. It has no path filter, scheduled latest-main input, or success-on-error path. Its read-only checkouts do not retain credentials.

scripts/daemon-compatibility.mjs reads e2e/compatibility.toml using Python 3.11+'s standard-library TOML parser. Duplicate keys and malformed TOML fail. The manifest must declare status = "ready", full daemon and committed-core SHAs, and all four required_test_targets in this order:

threads_e2e
threads_identity_invariants
threads_protected_intake
threads_terminal_recovery

current_threads_rev identifies the selected daemon's committed Git dependency, not the pull request's Threads checkout. Before installing the local override, the job checks that committed dependency against the manifest. The runner then proves the actual local dependency edge in the same way as observation.

The required job sets COVEN_THREADS_DAEMON_SUITE=compatibility. That profile requires an immutable COVEN_REF even on a scheduled/manual caller, proves all four source and Cargo test targets exist, and runs them in one locked, feature-enabled Cargo command. All existing source/config/lock guards apply before and after the command. Missing companions, failures, or final metadata drift fail the receipt. The receipt records the suite, all selected targets, and exact command; default advisory observations still select threads_e2e only. Artifacts remain unique to each run and attempt.

The manifest's ready state records pin eligibility, not a GitHub ruleset mutation or Phase-5 acceptance. Select a reviewed protected-main revision after normal downstream landing and exact-source revalidation. Only then add the new required context while preserving the existing strict checks. The old harness-required state must fail this job; it is not silently promoted and cannot fall back to main. Global deployed-history resolution, live Cave acceptance, and long-run reliability remain distinct obligations.

Expansion

  • run Linux/macOS/Windows on schedule;
  • run a three-journey live-daemon Cave project nightly;
  • promote one Chromium acceptance journey only after the flake budget is met;
  • set a coverage regression floor from measured baseline, not an invented target.

11. Workstream assignment

Lane Familiar/role Owns Must not own
T0 — contract and evidence Sage compatibility ledger, artifact schema, conformance mapping runtime implementation
T1 — core and regression Cody Rust lower-level tests, typed contracts, deterministic seams shared with daemon human sign-off
T2 — daemon boundary Nova integration lane process fixture, request client, SQLite/pending/filesystem assertions, restart redefining Threads semantics
T3 — audit/identity review Echo terminal-close, predicate-authority, replay and store invariants daemon lifecycle ownership
T4 — human acceptance Charm Cave live-daemon rendering/approval journeys client-side policy or approval authority
T5 — release/governance maintainer CI enforcement, protected branch, release evidence, rollback weakening gates for velocity

GitHub assignments remain on the accountable maintainer until bot/familiar identities have independently authenticated repository authority.

12. Success metrics

  • 8/8 critical authority journeys pass through the real daemon boundary;
  • 4/4 named Phase-5 implementation blockers carry red-to-green E2E evidence;
  • first-attempt E2E pass rate is at least 99.5% over 30 days;
  • flaky retry-successes still fail the required check;
  • pull-request critical-path p50 is under 8 minutes and p95 under 15 minutes;
  • duplicate or missing terminal close records remain zero;
  • correctness tests using real wall-clock sleeps remain zero;
  • every required run proves the current Threads checkout was linked;
  • escaped authority-boundary regressions remain zero.