Technical critique requested: evidence gate for autonomous coding runs

[camirian]

Fieldheld Recorder is asking for narrow technical critique of one public synthetic claim:

An autonomous coding-style run should leave enough evidence for a reviewer to decide whether to accept, reject, escalate, or block trust in the run.

Start here:

• Critique request: https://github.com/camirian/fieldheld-recorder/blob/main/docs/CRITIQUE_REQUEST.md
• Blocked sample: https://github.com/camirian/fieldheld-recorder/tree/main/samples/fieldheld-sample-run-blocked
• Run review checklist: https://github.com/camirian/fieldheld-recorder/blob/main/docs/RUN_REVIEW_CHECKLIST.md

Useful critique:

• Is the run intent specific enough to compare against the diff?
• Are inputs, actions, diff, verifier result, and review packet connected?
• Can a reviewer tell why a run was accepted, rejected, escalated, or blocked?
• Does the policy gate make clear that verification passing is not the same as trust being granted?
• Are any current claims stronger than the artifacts support?
• Which missing field would most improve review, replay, or rollback judgment?

Boundary: this is a synthetic local sample, not a live agent adapter, hosted service, compliance claim, production safety claim, or customer adoption claim.

[camirian]

Public discussion thread opened for review feedback:

https://www.reddit.com/r/LLMDevs/comments/1tshyc4/

HN note: the attempted Show HN submission appears to be filtered/dead for the new account, so the current active public feedback surface is Reddit plus this GitHub issue.

[camirian]

New review target added: the public sample now includes an adversarial corpus for hard-negative evidence-gate critique.

• Corpus: https://github.com/camirian/fieldheld-recorder/blob/main/samples/adversarial-corpus.json
• PR: Add adversarial review corpus #13

The verifier now checks 52 conditions, including coverage for verification failure, policy-sensitive change, forbidden effect, rollback overclaim, and verification overtrust.

[camirian]

New technical note for reviewers:

https://github.com/camirian/fieldheld-recorder/blob/main/docs/PASSING_TESTS_IS_NOT_ENOUGH.md

This is now the clearest entrypoint for the core claim: verification passing should not automatically mean trust is granted. It points to the adversarial corpus and the blocked policy-gate sample.

[camirian]

Issue #14 is now implemented:

• PR: Add vague intent adversarial sample #16
• New run: samples/fieldheld-sample-run-vague/
• Expanded corpus: https://github.com/camirian/fieldheld-recorder/blob/main/samples/adversarial-corpus.json

The public sample now includes five runs and 63 verifier checks, including traps for vague intent, unrelated diff, weak provenance, and stale evidence.

[camirian]

Update for reviewers landing here: I added a shorter entry path for critique.

Start with the reviewer packet index:
https://github.com/camirian/fieldheld-recorder/blob/main/docs/REVIEWER_PACKET_INDEX.md

For the trace/observability comparison:
https://github.com/camirian/fieldheld-recorder/blob/main/docs/TRACE_VS_EVIDENCE.md

The most useful objection is still specific: which trust decision in the public sample is not supported by the artifacts?

[camirian]

Update: Fieldheld Recorder now has a first concrete GitHub PR review mode.

Example:

fieldheld-recorder pr-review --repo camirian/fieldheld-recorder --pr 20

It writes a local reviewer bundle with PR metadata, changed files, risk summary, run record, artifact manifest, command log, and reviewer packet.

Merged PR:
#20

[camirian]

Update for reviewers: the PR-review workflow now has a concrete offline review target and an outcome worksheet.

Start here:

• Reviewer packet: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-sample/reviewer_packet.md
• Outcome worksheet: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-sample/review_outcome.json
• PR review bundle guide: https://github.com/camirian/fieldheld-recorder/blob/main/docs/GITHUB_PR_REVIEW_BUNDLE.md
• Latest merged slice: Add PR review outcome artifact #27

The sample can be regenerated and verified locally:

python3 -m fieldheld_recorder.sample
python3 -m fieldheld_recorder.verify_sample

Current verifier coverage: 75 checks.

Most useful critique question: would this PR review bundle reduce review burden, or does it add noise? If it adds noise, which artifact or field should be removed, changed, or made more explicit?

[camirian]

Narrower review ask is now available for direct outreach:

https://github.com/camirian/fieldheld-recorder/blob/main/docs/PR_REVIEW_CRITIQUE_REQUEST.md

It asks for a seven-minute review of only the deterministic PR-review fixture:

• reviewer packet: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-sample/reviewer_packet.md
• outcome worksheet: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-sample/review_outcome.json

Core question: would this reduce PR review burden, or does it add noise? If it adds noise, what is the first unsupported, noisy, or missing field?

[camirian]

Update for reviewers: the PR-review critique path now points to a reviewed self-demo fixture instead of only the unfilled synthetic worksheet.

Start here:
https://github.com/camirian/fieldheld-recorder/blob/main/docs/PR_REVIEW_CRITIQUE_REQUEST.md

Fast path:

• review request: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-self-demo/review_request.md
• reviewer packet: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-self-demo/reviewer_packet.md
• recorded outcome: https://github.com/camirian/fieldheld-recorder/blob/main/samples/fieldheld-pr-review-self-demo/review_outcome.json

The expected decision is needs_followup, not accept, because the recorded public status-check evidence is empty. That is the point of the fixture: the bundle should make the missing evidence visible instead of hiding it behind a generated summary.

Most useful critique question: does this seven-minute PR-review path reduce review burden, or does it add noise? If it adds noise, what is the first unsupported, noisy, or missing field?

[camirian]

Update for reviewers: the PR-review fixture matrix is now on main:

https://github.com/camirian/fieldheld-recorder/blob/main/docs/PR_REVIEW_FIXTURE_MATRIX.md

What changed:

• Low-risk docs fixture: accepted_for_review, recommended start accept.
• Adversarial green-check fixture: checks are successful, but vague intent plus high-risk file classes still produce needs_human_review / needs_followup.
• Self-demo fixture: filled review_outcome.json records needs_followup because status-check evidence is missing.
• Baseline generated fixture remains the unfilled worksheet path.

Core critique request: does this matrix help distinguish when a PR review bundle supports an accept-starting decision versus when it only supports follow-up, or does it add noise?

Boundary: public GitHub metadata and deterministic public-safe fixtures only. This does not approve PRs, execute CI, add a live adapter, or claim production readiness.