Skip to content

feat(evidence): publish the Flow 1.34.0 comparison; supersede 1.33.0 - #319

Merged
abrichr merged 1 commit into
mainfrom
feat/flow-134-evidence
Aug 29, 2026
Merged

feat(evidence): publish the Flow 1.34.0 comparison; supersede 1.33.0#319
abrichr merged 1 commit into
mainfrom
feat/flow-134-evidence

Conversation

@abrichr

@abrichr abrichr commented Aug 29, 2026

Copy link
Copy Markdown
Member

The reader is a reviewer of the public evals evidence pin, and anyone who quotes those numbers as current.

Flow 1.34.0 replaced 1.33.0 on PyPI. The daily evidence-freshness job went red (run 33253192977) with:

published evidence is pinned to openadapt-flow 1.33.0 but the current published release is 1.34.0

Those 1.33.0 numbers now describe an engine nobody serves. This follows #297 and #294: bounded campaigns, exact artifact binding, 1.33.0 superseded not deleted.

Counted result

93 counted trials against Flow 1.34.0, run locally (synthetic MockMed, loopback only, headless Chromium, no model calls):

Campaign Trials Result
Comparison 27 Compiled replay correct 9/9 across clean/theme/rename; selector controls halt safely under label drift
Independent replication 27 Same shape in a second complete run: compiled 9/9 correct
Transaction probe 30 Uncertain deliveries return RECONCILIATION_REQUIRED (18 of 30); no blind retry or replay dispatch
Remote lease safety 9 Absent-or-changed frame leases refuse actuation (HALTED_BEFORE_EFFECT, RECONCILIATION_REQUIRED)

Zero silent incorrect success, zero wrong actions, zero over-halts, zero model calls across all counted arms.

Every campaign states production_acceptance: false. This set is development evidence, not a Production lifecycle admission, and it establishes nothing about hosted, customer-workflow, native-desktop, RDP, or Citrix acceptance.

Environment

Python 3.12.7 on macOS arm64, openadapt-flow==1.34.0, openadapt-types==0.10.1, playwright==1.62.0, Playwright Chromium revision 1234. Those are the same Playwright and types versions the 1.33.0 set recorded. The Python patch is 3.12.7 rather than 3.12.13.

The wheel was fetched from PyPI and its SHA-256 checked against the published digest 56d32818… before any campaign ran. The Flow source is a clean checkout with v1.34.0 pointing at HEAD 30fc60e5.

Validation

  • live PyPI check passes: Published evidence docs/eval_results/current_flow_v1_34_0_local_20260829 is bound to the current openadapt-flow release 1.34.0
  • offline check passes
  • pytest tests/test_published_evidence_freshness.py: 37 passed

What a reviewer should check:

  • 1.33.0 remains on disk, marked superseded, stale_reason says it was not relabeled
  • the four campaigns bind wheel 56d32818…, sdist a941a700…, source 30fc60e5…
  • no production_acceptance boolean flipped to true

Opened by an agent session, not the founder.

The daily evidence-freshness job went red when Flow 1.34.0 replaced
1.33.0 on PyPI. This publishes a fresh local comparison, replication,
transaction-fault probe, and remote-lease probe bound to wheel
56d32818, sdist a941a700, and source 30fc60e5 (v1.34.0). The 1.33.0
set stays on disk and is marked superseded.

Opened by an agent session, not the founder.
@abrichr
abrichr merged commit 1ed4814 into main Aug 29, 2026
3 checks passed
@abrichr
abrichr deleted the feat/flow-134-evidence branch August 29, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant