Skip to content

fix(doc-review): cut review noise on plans, scope personas to doc shape#780

Merged
tmchow merged 2 commits intomainfrom
tmchow/doc-review-noise
May 5, 2026
Merged

fix(doc-review): cut review noise on plans, scope personas to doc shape#780
tmchow merged 2 commits intomainfrom
tmchow/doc-review-noise

Conversation

@tmchow
Copy link
Copy Markdown
Collaborator

@tmchow tmchow commented May 5, 2026

The problem

ce-doc-review has been a culprit of being too noisy and heavy-handed, especially when it runs at the end of ce-plan.

Adversarial and other persona reviewers were applying high-scrutiny techniques to routine plans, surfacing many "manual" findings that re-litigated decisions the brainstorm phase already settled. The post-plan flow forced a 4-option routing question into a per-finding walkthrough, producing a "review tax" that didn't match what users actually wanted after planning (which is to start work).

The goal

Make doc review feel right-sized:

  • Routine plans get a fast, silent autofix pass — no questions asked
  • Deep engagement is one click away when wanted
  • High-stakes content still gets full scrutiny
  • Reviewer personas apply the right techniques in the right phase (premise where premise lives, implementation where implementation lives)

Approach at a high level

1. Default to headless at ce-plan Phase 5.3.8

ce-doc-review now runs in mode:headless automatically at the end of ce-plan. In headless mode, safe_auto fixes (typos, count mismatches, broken refs, terminology drift, missing list entries) apply silently. gated_auto, manual, and FYI findings return as structured text — no walkthrough, no per-finding routing, no blocking prompts. The user sees a single summary line above the post-generation menu (Doc review applied 3 fixes. 2 decisions, 1 proposed fix remain.) and picks what to do next.

2. Deeper review is a first-class menu option, not a hidden free-form path

Added Run deeper doc review as option 2 in the post-generation menu so users can opt into the full interactive walkthrough explicitly. Renders as a 5-option numbered list per the AGENTS.md narrow exception when actionable findings remain; collapses cleanly to 4 options on AskUserQuestion when there's nothing actionable to walk through (FYI-only state included, since ce-doc-review's walkthrough is gated to anchor 75/100 gated_auto/manual findings and would otherwise dead-end).

3. Scope each reviewer persona by document shape

Previously every persona ran every technique on every doc, regardless of whether it was a brainstorm or a plan. That over-scrutinized plans (re-litigating premise the brainstorm already settled) and under-scrutinized requirements (running implementation feasibility checks on docs that intentionally have no implementation yet). Now:

  • ce-product-lens-reviewer — full 5 techniques on requirements-shape docs. On plan-shape docs with Origin: set, suppresses premise challenge and prioritization coherence; runs only strategic-consequences (when the plan introduces new strategic weight), implementation alternatives, and goal-requirement alignment.
  • ce-adversarial-document-reviewer — full 5 techniques on requirements-shape docs and origin-less plans. On plan-shape docs with Origin: set, suppresses premise challenging and simplification pressure entirely; runs only technical-assumption surfacing, technical-decision stress-testing, and architectural alternative blindness.
  • ce-scope-guardian-reviewer — full review on requirements. On plan-shape docs with Origin: set, focuses on implementation-time abstractions and scope-creep into deferred work; tightens the completeness principle to flag missing coverage only when origin demanded it.
  • ce-feasibility-reviewer — scopes shadow-path tracing, implementability, dependency analysis, and migration mechanics to plan-shape docs only. On requirements-shape docs, runs a tight "would this direction force a fundamental rework?" check.
  • ce-coherence-reviewer, ce-design-lens-reviewer, ce-security-lens-reviewer — light doc-type framing only (different identifiers and granularity expected per type), since they are net-positive at both levels.

4. Tighten the ce-adversarial-document-reviewer activation trigger

Dropped the "more than 5 distinct requirements or implementation units" criterion — that fired on virtually every Standard plan and was the biggest source of unwarranted dispatch. Activation now keys on real high-stakes signals: high-stakes domains (auth/payments/migrations), new abstractions, missing or extended origin, requirements-shape premise content, or explicit alternatives. Routine plans with structural rationale no longer trip adversarial.

5. Centralize document-type classification in the orchestrator

Moved doc-type classification into ce-doc-review using content-shape signals (frontmatter, R-IDs vs U-IDs, section structure) with file path as a tie-breaker hint only — path-based classification was brittle and the agent should read the doc. Added an Origin: slot to the subagent template so personas that adapt on origin read it authoritatively from a single source instead of re-parsing frontmatter themselves. One classification, all personas trust the slots.

Net effect

  • Routine plans dispatch the cheap minimum (ce-coherence-reviewer + ce-feasibility-reviewer)
  • High-stakes plans still dispatch the full team
  • The user is never asked to adjudicate findings from auto-runs unless they explicitly opt into the deeper review
  • Brainstorm review (when invoked) keeps full scrutiny — premise and assumptions are exactly what the brainstorm phase exists to validate

Test plan

  • bun run release:validate — clean
  • bun test — 1344 pass
  • Updated tests/pipeline-review-contract.test.ts to assert the new contract: headless default at ce-plan, deeper-review menu fixture, summary line above menu, FYI-only state hides the deeper-review option

🤖 Generated with Claude Code

Default ce-doc-review at ce-plan Phase 5.3.8 to mode:headless so safe_auto
fixes apply silently and remaining findings surface as a one-line summary
above the post-generation menu. Add Run deeper doc review as a first-class
menu fixture so interactive engagement is opt-in but obvious; hide it when
no actionable findings remain (FYI-only state included, since the
walkthrough is gated to anchor 75/100 gated_auto/manual findings and would
otherwise dead-end).

Tighten the adversarial activation trigger so routine plans no longer trip
it just for having structural rationale; activation now keys on real
high-stakes signals (high-stakes domains, new abstractions, missing or
extended origin, requirements-shape premise content, explicit alternatives).

Move document-type classification into the orchestrator using content-shape
signals (frontmatter, R-IDs vs U-IDs, section structure) with file path as
a tie-breaker only. Add an Origin: slot to the subagent template so personas
that adapt on origin (product-lens, adversarial, scope-guardian) read it
authoritatively instead of re-parsing frontmatter themselves -- one
classification, all personas trust the slots.

Per-persona doc-type adaptation:
- product-lens, adversarial, scope-guardian suppress premise/prioritization/
  simplification techniques on plan-shape docs with Origin: set, since the
  brainstorm validated those upstream
- feasibility scopes shadow-path tracing, implementability, and migration
  mechanics to plan-shape docs only; requirements-shape gets a tight check
  for fundamental-rework signals
- coherence, design-lens, security-lens carry light doc-type framing
  (different identifiers and granularity expected per type) since they are
  net-positive at both levels

Net effect: routine plans dispatch the cheap minimum (coherence + feasibility),
high-stakes plans dispatch the full team, and the user is never asked to
adjudicate findings from auto-runs unless they explicitly opt into the
deeper review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c828af5dcd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-plan/references/plan-handoff.md Outdated
- Restore chat-numbered-list fallback for the 5-option overflow menu when
  the platform's blocking question tool is unavailable or errors (Codex
  edit modes, Pi `ask_user` no-match), matching the AGENTS.md mandate to
  never silently skip a blocking question. Mirrored in SKILL.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@tmchow tmchow merged commit 8349e75 into main May 5, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant