Skip to content

[docs] Add AutoAgent production-gate evidence checklist#4443

Open
willblanchard wants to merge 2 commits intopaperclipai:masterfrom
willblanchard:paperclip/wat-1811-production-gate-checklist
Open

[docs] Add AutoAgent production-gate evidence checklist#4443
willblanchard wants to merge 2 commits intopaperclipai:masterfrom
willblanchard:paperclip/wat-1811-production-gate-checklist

Conversation

@willblanchard
Copy link
Copy Markdown

@willblanchard willblanchard commented Apr 24, 2026

Thinking Path

  • Paperclip orchestrates AI agents for zero-human companies.
  • Paperclip issue comments and review handoffs are the operating surface where PM, Meta Agent Manager, QE, and board users decide whether agent work is production-ready.
  • The WAT-1793 fake-pass audit showed that AutoAgent production-gate evidence can be hard to distinguish from development smoke evidence when agents write free-form comments.
  • Reviewers need a consistent checklist that names the exact production-gate artifacts, policy flags, and rejection conditions without requiring repo-internal knowledge.
  • This pull request adds that checklist to the board-operator docs and links it from execution-policy and agent communication guidance.
  • The benefit is faster, more consistent rejection of fake-pass production-gate evidence before it reaches merge or approval.

What Changed

  • Added a board-operator production-gate evidence checklist for AutoAgent review handoffs.
  • Listed required evidence: command, run ID, summary/result artifacts, git SHA, dirty state, bronze/silver/gold/OpenEI counts, canary policy flags, and gate class.
  • Added standard rejection language for development-smoke evidence, hidden skip flags, untested silver, missing Mongo upload, missing independent review, and missing artifacts.
  • Linked the checklist from execution-policy and comments/communication docs.

Verification

  • git diff --check
  • node -e "JSON.parse(require('fs').readFileSync('docs/docs.json','utf8')); console.log('docs/docs.json valid')"
  • pnpm run check:tokens
  • PR CI is expected to rerun after the latest docs-only push.

Risks

  • Low risk: docs-only change.
  • The WAT-1793 reference is plain text to avoid a broken absolute link in the published docs site.

For core feature work, check ROADMAP.md first and discuss it in #dev before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See CONTRIBUTING.md.

Model Used

  • OpenAI GPT-5.5 via Codex local adapter, reasoning effort medium, with shell/GitHub/Paperclip tool use.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have specified the model used (with version and capability details)
  • I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
  • I have run tests locally and they pass
  • I have added or updated tests where applicable
  • If this change affects the UI, I have included before/after screenshots
  • I have updated relevant documentation to reflect my changes
  • I have considered and documented any risks above
  • I will address all Greptile and reviewer comments before requesting merge

Paperclip

  • WAT-1811

@willblanchard
Copy link
Copy Markdown
Author

@Praviwwattshift please review PR #4443 when you have a chance. CEO attempted the formal reviewer request, but both the GitHub connector and gh pr edit --add-reviewer Praviwwattshift are blocked by GitHub permission RequestReviewsByLogin for the current account/integration.

Readiness state at request time:

  • Head: 7d2b8c399aaa54556485d06d3f90bfbbe4dd4678
  • policy, e2e, and Snyk are green
  • verify is still in progress on Actions run 24914077684 / job 72962268855

Maintainer action still needed: formally request Pravin as reviewer, or have Pravin review directly from this mention once verify is green.

@willblanchard willblanchard marked this pull request as ready for review April 24, 2026 22:45
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 24, 2026

Greptile Summary

This docs-only PR adds a production-gate evidence checklist for AutoAgent review handoffs and links it from the execution-policy and comments-and-communication guides. The content and structure are clear, but the PR description is missing the required Thinking Path and Model Used sections mandated by CONTRIBUTING.md — please add them before merging.

  • The internal link [WAT-1793](/WAT/issues/WAT-1793) on line 10 of the new checklist page will likely 404 in the published Mintlify docs; replace it with the full Jira URL or plain text.

Confidence Score: 4/5

Safe to merge after fixing the broken Jira link and adding the required PR description sections.

One P1 finding (likely-broken internal Jira link in published docs) and two required PR-description sections missing per CONTRIBUTING.md (Thinking Path, Model Used) keep this from a 5. All other changes are straightforward docs additions with correct relative paths.

docs/guides/board-operator/production-gate-evidence.md — Jira link on line 10 needs to be a full URL or plain text.

Important Files Changed

Filename Overview
docs/guides/board-operator/production-gate-evidence.md New checklist page with clear structure; contains an internal Jira link (/WAT/issues/WAT-1793) that will likely 404 in the published docs site.
docs/docs.json Adds production-gate-evidence to the Board Operator nav group. Pre-existing duplicate entries for execution-workspaces-and-runtime-services and delegation remain unfixed but were not introduced by this PR.
docs/guides/agent-developer/comments-and-communication.md Adds a one-line cross-reference to the new production-gate-evidence page; relative path is correct.
docs/guides/execution-policy.md Adds a short paragraph linking to the new checklist; relative path resolves correctly.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: docs/guides/board-operator/production-gate-evidence.md
Line: 10

Comment:
**Internal Jira link may not resolve in published docs**

The link `[WAT-1793](/WAT/issues/WAT-1793)` uses an absolute path that is unlikely to resolve in a Mintlify-hosted docs site. If this Jira project isn't publicly accessible or the docs host doesn't proxy `/WAT/...`, readers will hit a 404. Either replace it with the full Jira URL (e.g. `https://paperclip.atlassian.net/browse/WAT-1793`) or remove the hyperlink and leave the ticket ID as plain text.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docs/guides/board-operator/production-gate-evidence.md
Line: 1

Comment:
**PR description is missing required sections per CONTRIBUTING.md**

`CONTRIBUTING.md` requires every PR to include a **Thinking Path** (explaining the reasoning from project context down to this specific change) and a **Model Used** section (listing the AI model or "None — human-authored"). The current description has a Summary and Validation block but is missing both of these required sections. Please add them before merging.

**Context Used:** CONTRIBUTING.md has a guide for a good PR message ... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "docs: add production gate evidence check..." | Re-trigger Greptile


This checklist comes from the fake-pass risk audit in [WAT-1793](/WAT/issues/WAT-1793). It is intentionally stricter than normal progress comments: a production gate can pass only when the evidence shows the real production path, complete inputs, persisted outputs, and independent review.

## Required Evidence
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Internal Jira link may not resolve in published docs

The link [WAT-1793](/WAT/issues/WAT-1793) uses an absolute path that is unlikely to resolve in a Mintlify-hosted docs site. If this Jira project isn't publicly accessible or the docs host doesn't proxy /WAT/..., readers will hit a 404. Either replace it with the full Jira URL (e.g. https://paperclip.atlassian.net/browse/WAT-1793) or remove the hyperlink and leave the ticket ID as plain text.

Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/guides/board-operator/production-gate-evidence.md
Line: 10

Comment:
**Internal Jira link may not resolve in published docs**

The link `[WAT-1793](/WAT/issues/WAT-1793)` uses an absolute path that is unlikely to resolve in a Mintlify-hosted docs site. If this Jira project isn't publicly accessible or the docs host doesn't proxy `/WAT/...`, readers will hit a 404. Either replace it with the full Jira URL (e.g. `https://paperclip.atlassian.net/browse/WAT-1793`) or remove the hyperlink and leave the ticket ID as plain text.

How can I resolve this? If you propose a fix, please make it concise.

@@ -0,0 +1,69 @@
---
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 PR description is missing required sections per CONTRIBUTING.md

CONTRIBUTING.md requires every PR to include a Thinking Path (explaining the reasoning from project context down to this specific change) and a Model Used section (listing the AI model or "None — human-authored"). The current description has a Summary and Validation block but is missing both of these required sections. Please add them before merging.

Context Used: CONTRIBUTING.md has a guide for a good PR message ... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/guides/board-operator/production-gate-evidence.md
Line: 1

Comment:
**PR description is missing required sections per CONTRIBUTING.md**

`CONTRIBUTING.md` requires every PR to include a **Thinking Path** (explaining the reasoning from project context down to this specific change) and a **Model Used** section (listing the AI model or "None — human-authored"). The current description has a Summary and Validation block but is missing both of these required sections. Please add them before merging.

**Context Used:** CONTRIBUTING.md has a guide for a good PR message ... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))

How can I resolve this? If you propose a fix, please make it concise.

@willblanchard
Copy link
Copy Markdown
Author

Addressed Greptile findings on head f41b358a4357341b97931657840f4cbd0a1e35a7: removed the published-docs-breaking absolute WAT link while keeping the WAT-1793 issue reference as plain text, and updated the PR description to include Thinking Path and Model Used. Fresh checks are green: policy, e2e, verify, and Snyk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant