Skip to content

feat(evidence): add v3 qualification and immutable recovery - #303

Merged
abrichr merged 10 commits into
mainfrom
codex/evals-release-lifecycle-contract
Aug 29, 2026
Merged

feat(evidence): add v3 qualification and immutable recovery#303
abrichr merged 10 commits into
mainfrom
codex/evals-release-lifecycle-contract

Conversation

@abrichr

@abrichr abrichr commented Aug 26, 2026

Copy link
Copy Markdown
Member

What this changes

This PR adds the private v3 workflow qualification producer, fixes two immutable release recovery gaps, and updates the public evidence layer to the final contracts at OpenAdaptAI/.github@989681f6f475616b7e2cb72360c716db0927f7ad:

  • qualification release v2
  • qualification admission v4
  • qualification evidence decision receipt v2
  • production acceptance manifest and summary v3
  • Flow release verification receipt v1

The private producer covers healthy, safe halt, idempotency replay, uncertain delivery, declared attended, and governed repair for every task. It requires at least three signed trials in each task and condition cell. It derives silent incorrect success, over-halt, dispatch, retry, replay, effect, cleanup, intervention, and repair counts from signed receipt facts.

Every uncertain-delivery trial must return RECONCILIATION_REQUIRED. The producer requires one dispatch for that trial and rejects blind retry or replay dispatch. Each receipt also binds its source artifact, campaign, run, protected repository, workflow, ref, commit, environment, and runner identity.

The public summary command uses the current wall clock and a clean checkout of the pinned central commit. An accepted summary requires an active signer registry, authority state, and revocation state, plus registered objects that pass the central verifier. This pin's signer registry pointer is inactive, so the command refuses to write accepted evidence.

The pair command validates the regular object against its exact central JSON Schema before it writes a file. Kinds without a final central schema are refused.

The PyPI recovery check reads an existing version before skip-existing can continue a partial upload. GitHub Release publication uses a repository-scoped openadapt-release token.

This PR installs .github/workflows/issue-production-acceptance.yml at the path the central schemas name as the only issuer. Every run refuses. It does not write a manifest, a summary, an attestation, or a GitHub Release.

Authority and privacy boundary

Evals does not own signing keys, the signer registry, the revocation ledger, or KMS policy. The command imports the verifier from the exact pinned central checkout and refuses a dirty checkout.

Private campaign rows, source artifacts, task names, application values, environment values, and live identities stay behind the private evidence boundary.

Fail-closed dependencies (not waits)

Central main at 989681f6f475616b7e2cb72360c716db0927f7ad contains the final schemas and verifier code. Its signer registry pointer is inactive. The summary and pair commands refuse to write accepted evidence at this pin. That is the correct production behavior.

Checked against live state:

  • signer registry, authority state, revocation state: still inactive on OpenAdaptAI/.github main. The producer refuses. No mock accepted evidence.
  • production acceptance issuer: installed in this PR and refuses issuance.
  • protected trial receipt vectors / source manifest v2 / source seal v2: not present in the public central pin. The public path cannot accept evidence without them because the registry is inactive.
  • protected release environments and App credentials: release-identity and pypi exist; OPENADAPT_RELEASE_APP_ID=4730708; OPENADAPT_RELEASE_APP_PRIVATE_KEY is set in both environments.
  • hosted test must pass at the exact head after the issuer commit.

This PR does not issue an admission, activate an issuer, sign evidence, publish a package, create a release, change a repository setting, or copy private evidence into a public repository.

@abrichr
abrichr force-pushed the codex/evals-release-lifecycle-contract branch from b3dd56f to 5b56880 Compare August 27, 2026 16:52
@abrichr
abrichr changed the base branch from main to release/next August 27, 2026 16:52
Base automatically changed from release/next to main August 27, 2026 16:56
@abrichr
abrichr force-pushed the codex/evals-release-lifecycle-contract branch from 5b56880 to 6225d0a Compare August 27, 2026 17:00
@abrichr
abrichr force-pushed the codex/evals-release-lifecycle-contract branch from d7383d3 to 5a95ad4 Compare August 27, 2026 22:25
The pinned digest covers the canonical JSON of production_acceptance_policy().
Rebasing onto main combined main's policy with this branch's manifest_schema
bump, so neither the old branch pin nor main's pin is correct for the result.

Recomputing the policy on both sides shows exactly one field differs:

    manifest_schema: openadapt.production-acceptance/v1 -> /v2

which is this branch's intended change. The new pin is the digest of that
policy.
@abrichr
abrichr force-pushed the codex/evals-release-lifecycle-contract branch from bd16ad3 to 3a98c78 Compare August 28, 2026 20:15
@abrichr abrichr changed the title ci: verify immutable eval release recovery feat(evidence): add v3 qualification and immutable recovery Aug 28, 2026
@abrichr
abrichr force-pushed the codex/evals-release-lifecycle-contract branch from 3a98c78 to 984a885 Compare August 28, 2026 20:18
The central schemas name this path as the only issuer. The workflow
exists and refuses. It does not write accepted evidence.
@abrichr
abrichr marked this pull request as ready for review August 29, 2026 14:47
@abrichr
abrichr merged commit 5901a64 into main Aug 29, 2026
1 check passed
@abrichr
abrichr deleted the codex/evals-release-lifecycle-contract branch August 29, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant