Skip to content

Commit 6c860f7

Browse files
mikhail-dclclaude
andauthored
chore: improve AI PR-review (model selection, cost reporting, instructions) (#9063)
* chore: consolidate PR review feedback into AI review instructions Fold the design-level review of #9059 into .github/prompts/review-instructions.md so the automated reviewer catches architectural defects, not just line-level bugs: - Add a DESIGN & INTEGRATION CHECK, run first, with a mandatory owner search (name the *Load*/*Unload*/facade/disposal owners that already own the lifecycle), anti-rationalization guards (author framing, "it's hard", and assembly boundaries are not justifications), and a teardown/consumption trace for every subscription/buffer. - Require judging whether a new long-lived unit should exist at all vs. integrating into the existing lifecycle owner. - Add blocking categories: subscription/resource leaks, allocated-but-unconsumed infrastructure, detached async for essential work, nullability-contract violations, dead/false-intent conditions. Verified clean-room against #9059: reproduces the human review's architectural findings reliably on Opus and the resource/lifecycle findings on Sonnet. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: add construction/encapsulation/resource checks to AI review instructions Consolidates a manual PR review of the media-stream camera-off placeholder work into .github/prompts/review-instructions.md, complementing the existing DESIGN & INTEGRATION CHECK. - Add a CONSTRUCTION, ENCAPSULATION & RESOURCE CHECKLIST: DI vs self-new, invariant-at-construction, naming-vs-responsibility, AI-comment smell, encapsulation/responsibility leaks, side-effect-free funcs, magic values + MordorConstants reuse, thread-safety docs, and resource lifecycle (use-after-destroy ownership, cache cost/benefit, all-or-nothing eviction, per-frame GPU expense). - Reframe the reporting guardrail so project-standard violations are blocking, not "nice-to-have" nitpicks (this was suppressing exactly these findings). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: make @claude review model-selectable and report run cost On-demand `@claude review` now accepts a model, defaulting to Sonnet: - "@claude review opus" runs claude-opus-4-6; otherwise claude-sonnet-4-6. - Model-aware --max-turns ceiling (Sonnet 50, Opus 30) to bound spend. A capped run fails as error_max_turns, so caps are sized to let a normal review finish rather than pay-and-truncate. - The selected model is shown in the pending status. Both the on-demand and auto-review jobs append the run's USD cost to the end of the review comment (after the QA_REQUIRED block), e.g. "Review cost: $0.42 - claude-opus-4-6 - 28 turns". Runs on always() so failed/aborted reviews still report cost (annotated "did not finish"), is idempotent across re-runs, falls back to a standalone comment if the bot comment can't be edited, and is continue-on-error so it never fails the job. The push-triggered auto-review job stays on claude-sonnet-4-6. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: sharpen AI review predicate/accessor checks with a mandatory member audit Consolidates NickKhalow's second review round on the media-stream PR (#9043) into the AI review instructions and sharpens the weakest checks. - Naming: flag names that hide a precondition the member depends on (e.g. IsCurrentVideoMuted -> IsCurrentCameraVideoMuted). - Comments: extend the smell list to over-explanation that restates a well-named member and assumptions about code outside the scope. - Add a Predicates, accessors & single-use members section, framed as a MANDATORY MEMBER AUDIT (enumerate each touched member's consumers, then test it against single-use-merge, absent-not-false/null, and don't-re-derive), mirroring the existing OWNER SEARCH pattern to force systematic coverage instead of opportunistic line-by-line flagging. - Carve thin forwarding accessors (e.g. fromIdentity => key.identity) out of the single-use rule so it does not recommend deleting intentional encapsulation — a false positive caught during local validation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: extract Claude review cost reporting into a shared script The cost-reporting github-script block was duplicated verbatim in the auto-review and on-demand-review jobs. Move it into .github/scripts/report-review-cost.js, called from both via require(), so there is a single copy and it can be run locally with a stubbed github/context. Addresses NickKhalow review feedback on #9063. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: restructure AI review instructions into an ordered step protocol Keep the single-agent flow and one parsed verdict, but split the work into an explicit STEP 1..9 protocol (load context, root-cause, design/owner search, member audit, line-level review, complexity, QA, warnings, verdict). Ordering is now structural with a required written output per step, rather than the soft "do this before any line-level review" wording the reviewer distrusted. The verdict trailer is unchanged, so the auto-approve/QA automation is unaffected. Also folds in the rest of NickKhalow's review feedback on #9063: - enumerate concrete leak openers and their mirrors in the leak check - add RAII-for-native-resources to the resource-lifecycle checks - split the dead/false-intent check: keep the semantic "misleading intent" part, leave unreachable / always-true-false to ReSharper Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: let the AI reviewer search the repo with Read/Glob/Grep Add Read, Glob, and Grep to --allowedTools in both the auto-review and on-demand review jobs so the reviewer can run ripgrep-style searches. This is additive to the action's default read-only file access; Bash stays limited to the gh allowlist and the auto-approve path is unchanged (the new tools are read-only, and approval is a separate parsed step). Wire the instructions to use it: Step 1 states the working tree is checked out with Read/Glob/Grep available; Step 5's leak check (#7) now greps each changed *.cs for the openers and their mirrors. Addresses NickKhalow's grep-driven-leak-detection feedback on #9063. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent a8779e0 commit 6c860f7

3 files changed

Lines changed: 190 additions & 24 deletions

File tree

.github/prompts/review-instructions.md

Lines changed: 92 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,102 @@
1-
Read `CLAUDE.md` and `docs/README.md` before reviewing. Load the relevant subsystem doc for the diff. Cite the specific rule or doc section when flagging a violation.
1+
# PR REVIEW PROTOCOL
22

3-
--- ROOT-CAUSE CHECK ---
4-
Before listing issues, state in the summary comment: what problem is this PR solving, and does the diff fix the cause or a symptom?
5-
Flag as FAIL if the diff null-checks a value that shouldn't be null there, swallows an exception without addressing the source, works around a race instead of fixing ordering, or disables a check to make tests pass. Say what the actual fix would be.
3+
You are the single automated reviewer for this PR. Execute STEP 1 → STEP 9 below **in order, as one pass**, and record each step's required output in your summary comment before moving on. The ordering is the method, not a suggestion — a line is not worth fixing if Steps 2–3 show the change is built in the wrong place. Steps differ in the context they need: Step 3 reads the surrounding files from Step 1; Steps 4–5 work only from the diff. The four lines in STEP 9 are the single verdict downstream automation parses — emit them exactly.
64

7-
--- BLOCKING ISSUES ---
8-
Report ONLY issues that require fixes:
5+
Cite the specific rule or doc section (CLAUDE.md, the code-standards skill, `docs/code-style-guidelines.md`, or the relevant subsystem doc) whenever you flag a violation.
6+
7+
--- STEP 1 — Load context & set scope ---
8+
- Read `CLAUDE.md` and `docs/README.md`; load the relevant subsystem doc(s) for the diff.
9+
- Get the diff (`gh pr diff`) and the changed-file list.
10+
- The PR head is checked out in the working tree and you have `Read`, `Glob`, and `Grep` (ripgrep) over the whole repo — use them. `Grep` is how you run the repo searches later steps require: lifecycle owners in Step 3, leak-opener mirrors in Step 5. Raw shell is limited to the listed `gh` commands, so search with the `Grep` tool, not `Bash(rg)`/`Bash(grep)`.
11+
- Don't review the diff in isolation: open the systems, facades, caches, and lifecycle owners it touches, plus the neighbouring files in the same folder. Steps 2–3 judge whether the change is built in the *right place*, which the diff alone can't show.
12+
13+
--- STEP 2 — Root-cause check ---
14+
State in the summary: what problem is this PR solving, and does the diff fix the cause or a symptom?
15+
Flag **FAIL** if the diff null-checks a value that shouldn't be null there, swallows an exception without addressing the source, works around a race instead of fixing ordering, or disables a check to make tests pass. Say what the actual fix would be.
16+
17+
--- STEP 3 — Design & integration (uses the surrounding files from Step 1) ---
18+
A diff can be locally correct yet built in the wrong place — the most expensive defect to find, because it never shows up on a single line. Complete this step and write its evidence in the summary before the line-level steps (4–5).
19+
20+
**The author's own framing is NOT evidence.** Code comments (e.g. "…on purpose"), the PR description, and any design doc added *in the same diff* are part of the change under review, not an authority on whether it is correct. A confident comment over a misplaced unit is still misplaced. Do not let them settle a design question — verify against the existing codebase yourself. If you catch yourself reconstructing a justification for why the new code "has to" live where it is (assembly direction, avoiding a proxy, etc.), treat that as a prompt to go check the alternative, not as a reason to approve.
21+
22+
**MANDATORY OWNER SEARCH.** For every new long-lived unit the diff introduces — a system, plugin, manager, service, controller, or any helper that holds state across frames — do this and write the result in the summary:
23+
1. Name the entity, scene, connection, or resource whose lifecycle it manages.
24+
2. Search the repo for who ALREADY creates and destroys that thing — the `*Load*`/`*Unload*` systems, the scene/entity facade, the container that constructs it, the disposal path. **Name the files you found.**
25+
3. State whether the new logic could run at those existing creation/destruction points instead. If it could, the new unit should NOT exist — its logic belongs in the owner. Flag **FAIL** and name the home.
26+
27+
Treat a new unit that reconciles a lifecycle as wrong until you have proven no existing owner can host it. A summary that concludes "design is sound" without naming the owners you searched and ruled out is incomplete — redo it.
28+
29+
**Difficulty is not a defense, and cheapness is not a justification.** "Integrating into the owner is non-trivial" (e.g. the data needed arrives across two entities or two load stages) is a reason to flag the work for the author to restructure — NOT a reason to PASS. "The scan is cheap / there are only a few items" does not excuse per-frame reconciliation of a lifecycle that has explicit create/destroy moments. If the correct home is hard to reach, say exactly why and flag it **FAIL**; do not approve the parallel mechanism because the right design takes more work.
30+
31+
**Module/assembly boundaries are part of the design under review, not fixed constraints.** If the only thing stopping the natural lifecycle owner from doing the work is that its assembly does not reference the needed code, that boundary is itself the smell — flag it, do not cite it as justification. "The current asmdef graph forces this placement" means the integration was never designed; the feature likely needs restructuring so the work lives with its owner (made part of the load / facade / disposal path), even if that means changing the dependency graph or moving the seam. A parallel reconciler that exists *because* the owner's assembly can't reach the code is a **FAIL**, not an acceptable workaround.
32+
33+
Flag as **FAIL** (these are blocking design issues, not nitpicks) when new code:
34+
- **Duplicates a lifecycle that is already owned.** It connects/registers when something appears and disconnects/removes when it disappears, while creation and destruction already have explicit owners (scene load/unload, entity creation / `DeleteEntityIntention`, portable-experience or asset disposal, facade construction). The wiring belongs in that owner; the new unit usually should not exist.
35+
- **Reconciles every frame what is known at an explicit moment.** An `Update()` that re-queries or rescans collections each frame to detect what appeared or disappeared, when those moments are explicit elsewhere. Connect/subscribe at the creation moment; disconnect/remove at the disposal moment — not by polling each tick.
36+
- **Reconciles by scanning against a "live set".** A retain-only / keep-only-what-is-still-here pass over a collection when the disposal moment is explicit. Remove the specific entry then (`Remove(id)`) instead of scanning to discover what is stale.
37+
- **Holds persistent state outside ECS.** A system or its helper owns persistent collections that mirror an entity/scene lifecycle. Per-frame scratch buffers cleared each tick are fine; persistent membership belongs in ECS or the lifecycle owner (CLAUDE.md §1).
38+
- **Reaches data through a repeated intermediary lookup** instead of the canonical source — e.g. checking the current scene's state through an injected helper that re-resolves a dictionary on every call, rather than the scene facade obtained from the scene cache. Drive the check from the owner that already holds the data.
39+
40+
**TEARDOWN / CONSUMPTION TRACE.** For every subscription, event/callback hookup, connection, room, buffer, or measurement the diff adds, point to the exact line that unsubscribes / disposes / consumes it. If you cannot find that line, flag it — a missing unsubscribe is a leak; a buffer or measurement that is written but never read is dead infrastructure. Do not assume a `Dispose()` elsewhere removes a subscription unless you can see it.
41+
42+
--- STEP 4 — Member audit (works from the changed members) ---
43+
For every public property or accessor the diff adds or changes, find its consumers (a member is usually called within its own class) and state the count in your summary. Then test it against the failure modes below — a member you list as touched but do not audit is an incomplete review:
44+
- **Single-use → merge or inline.** A property used by exactly one consumer, or one that re-validates state its sole caller already guarantees, should be merged into that consumer and named by intent (e.g. an `IsCurrentVideoMuted` used only inside `IsShowingPlaceholder` → a single `ShouldShowPlaceholder`), or made a private method that takes the already-validated non-null instance. Don't keep a single-use intermediate that re-checks invariants (CLAUDE.md §11 "Extracting when you should merge"). This targets *derived* predicates that combine or re-check state — NOT a thin forwarding accessor that centralizes access to one field of an owned object (e.g. `fromIdentity => key.identity`); those are legitimate encapsulation even when single-use, and callers should route through them rather than reach past them.
45+
- **Absent ≠ false/null.** A predicate or accessor that returns a default (`false`, `null`) for the absent / not-applicable case conflates "no" with "undefined" (e.g. `IsMuted` returning `false` when there is no video to be muted at all, so the question is meaningless rather than answered "no"). Make it a method taking the required non-null state (`IsMuted(VideoStreamInfo video)`) so the absent case is handled by the caller at the one place it is known.
46+
- **Don't re-derive what already exists.** If an existing property/method already yields the value or does the same lookup, call it — don't open-code the condition again (e.g. reuse `IsShowingPlaceholder` instead of repeating its check; use the dedicated `fromIdentity` accessor instead of re-reading the field).
47+
- **Redundant guard.** A guard that repeats a condition already guaranteed earlier in the same flow — remove it (see Step 5 issue 11).
48+
49+
--- STEP 5 — Line-level review (works from the diff) ---
50+
Report ONLY issues that require fixes. Make two passes over the changed lines.
51+
52+
**A. Blocking-issue categories**
953
1. Code quality violations per CLAUDE.md standards (cite the rule)
1054
2. Bugs or potential runtime errors
1155
3. Security vulnerabilities
1256
4. Performance issues
1357
5. Missing error handling
1458
6. Unclear or problematic logic
59+
7. Resource / subscription leaks — an acquisition without a matching teardown at the corresponding disposal point. Scan for the concrete openers and confirm each has its mirror: `Subscribe(``Unsubscribe`, `AddListener(``RemoveListener`, `+=` on an event/`Action``-=`, a pool `Get(`/`Rent(``Release`/`Return`, `new CancellationTokenSource(``Cancel`+`Dispose`, a `Connect`/room/handle/`IDisposable` open→`Dispose`. Do this with `Grep`: search each changed `*.cs` file for the openers, then grep the same file/type for the matching mirror, and flag any opener whose mirror is missing.
60+
8. Allocated-but-unconsumed infrastructure — buffers, measurements, events, or caches that are populated/written but never read anywhere in the diff or the codebase.
61+
9. Detached async for essential work — `.Forget()` or fire-and-forget `UniTaskVoid` that performs setup the feature depends on. Essential async must be awaited inside the relevant lifecycle, not left detached (CLAUDE.md §9).
62+
10. Nullability-contract violations — assigning `null` or a maybe-null value to a non-nullable declaration, or a defensive null-check against a non-nullable declaration. Both lie about what can be null (CLAUDE.md anti-patterns).
63+
11. False-intent conditions — a check that is technically reachable but conveys a misleading intent for the case being added (e.g. a connection check left in place for a case it can never describe). (Purely unreachable / always-true-or-false conditions are left to ReSharper, which already enforces them in CI via the lint hook — don't spend review budget re-deriving them.)
64+
65+
**B. Design, encapsulation & resource smells** — zoom in from Step 3 (which asks whether the unit belongs at all) to how the units the diff touches are built, named, and manage memory. Each looks minor in isolation, so a shallow pass misses it, but the project treats these as required fixes.
66+
67+
*Construction & dependency injection*
68+
- A class that takes raw materials and `new`s up its own collaborator instead of receiving the built dependency. Collaborators are constructed at the composition root and injected; constructors take dependencies, not ingredients.
69+
- One invariant spread across a lazy `Ensure...()` plus null-guards on the fields it sets, so the instance can exist partially initialized. Establish the invariant in the constructor or a factory so the object is never half-built.
70+
71+
*Naming & comments*
72+
- A type/member name that describes a mechanism or a moment rather than the responsibility it owns (e.g. a `...Composer` / `...Helper` / `...Manager` that is really the source of one specific thing). Flag names that don't match what the class actually provides.
73+
- A name that hides a precondition the member actually depends on (e.g. a flag that only applies to camera video, not screenshare, named `IsCurrentVideoMuted` instead of `IsCurrentCameraVideoMuted`). The implicit condition belongs in the name.
74+
- Comments that state the obvious, restate what a well-named member already says, narrate caller/external behavior, assume things about code outside this scope's responsibility, or read as AI scaffolding (e.g. "must be verified in the editor — cannot be validated headlessly"). A comment must explain only what the annotated code itself does or guarantees; over-explanation is a defect, not thoroughness.
75+
76+
*Encapsulation & responsibility*
77+
- Behavior implemented in class A out of data that belongs to class B (e.g. a system composing/blitting a texture the owning player should provide). Move the behavior to the owner.
78+
- Public state exposed only so another class can replicate logic that belongs with that state — once the behavior moves to the owner, the exposed property is redundant. Flag the leak.
79+
- A method that mutates a shared field as a side effect (e.g. sets a `...Failed` flag) when it could just return the result. Prefer returning a value over hidden state changes.
80+
81+
*Constants & magic values*
82+
- Inline numeric / color / position literals that should be named constants declared at the top of the type (member-ordering: consts first).
83+
- A hand-rolled value that duplicates an existing shared constant (e.g. a far-away "parking" position when `MordorConstants` already defines one). Reuse it.
84+
85+
*Nullability & thread-safety*
86+
- A `T?` dependency with a null-handling branch: confirm absence is a legitimate runtime state, not a wiring bug that should make the field non-nullable (pairs with issue 10).
87+
- A stateful class with caches or mutable fields whose XML summary omits its threading contract. Require an explicit note (e.g. "not thread-safe").
1588

16-
For each issue found:
17-
- Location: File and line number
18-
- Problem: What is wrong (be specific)
19-
- Fix: Exact change needed
20-
- Why: Brief explanation of the impact
89+
*Resource lifecycle & memory (GPU textures, pools, caches)* — distinct from the subscription/teardown leak checks (Step 3's trace and issue 7, which catch opened-but-never-closed); these catch resources destroyed while still referenced, or that cost more than they save:
90+
- Ownership / lifetime: a method that returns a reference to a resource it later `Release()`/`Destroy()`s on its own schedule, with no ownership or refcount contract, is a use-after-destroy hazard — it survives only because today's single caller consumes the result the same frame. Flag references a caller might hold past their valid lifetime.
91+
- Cost vs. benefit: estimate what a cache holds (e.g. N × width × height × bytes-per-pixel of VRAM) against what recomputation costs. A cache that holds significant memory but buys little (cheap to recompute, result copied immediately) should be a single reusable target or recomputed on change, not a per-key cache.
92+
- Eviction: clearing the whole cache when it fills (all-or-nothing) instead of bounded LRU / single-slot reuse.
93+
- Per-frame expense: a GPU/CPU-heavy op (`Blit`, `Render`, allocation) run every frame for a result that changes only on an event — move it to the event.
94+
- RAII for native resources: a native handle (`RenderTexture`, `NativeArray`, `ComputeBuffer`, room/connection, `IDisposable`) whose release is a manual `Release()`/`Dispose()` reachable only on the happy path — an early return or exception leaks it. Prefer `using`, or ownership by a type whose own `Dispose()` frees it, so the release cannot be skipped.
2195

22-
Use mcp__github_inline_comment__create_inline_comment for specific line issues.
23-
Use Bash(gh pr comment) for a top-level summary comment.
24-
Do NOT include praise, style preferences, or nice-to-have suggestions.
96+
**Reporting (Steps 3–5).** For each issue: Location (file and line), Problem (be specific), Fix (exact change needed), Why (brief impact). Use `mcp__github_inline_comment__create_inline_comment` for specific line issues and `Bash(gh pr comment)` for the top-level summary. Do NOT include praise or subjective style opinions. But a violation of a project standard is NOT a "nice-to-have": naming, magic numbers, member ordering, encapsulation, and resource ownership are required by CLAUDE.md / the code-standards skill / `docs/code-style-guidelines.md`. Report them as blocking and cite the rule — do not soften them into optional suggestions or skip them as nitpicks.
2597

26-
--- COMPLEXITY ASSESSMENT ---
27-
After reviewing the diff, classify the PR complexity as SIMPLE or COMPLEX.
98+
--- STEP 6 — Complexity assessment ---
99+
Classify the PR complexity as SIMPLE or COMPLEX.
28100

29101
A PR is SIMPLE when ALL of the following are true:
30102
- Touches 3 or fewer files with under ~150 lines of meaningful changes
@@ -56,7 +128,7 @@ A PR is COMPLEX when ANY of the following are true:
56128
- Risk of silent data corruption or subtle regressions is non-trivial
57129
- Large diff (4+ files or significant logic changes)
58130

59-
--- QA ASSESSMENT ---
131+
--- STEP 7 — QA assessment ---
60132
Determine whether QA (manual testing) is needed for this PR.
61133

62134
QA_REQUIRED: NO when ALL of the following are true:
@@ -71,13 +143,14 @@ QA_REQUIRED: YES when ANY of the following are true:
71143
- Touches UI, rendering, avatars, scenes, networking, or player-facing systems
72144
- Changes asset loading, authentication, or navigation flows
73145

74-
--- NON-BLOCKING WARNINGS ---
146+
--- STEP 8 — Non-blocking warnings ---
75147
Emit these as warnings in the summary comment. They do NOT cause a FAIL on their own.
76148

77149
- **Main Scene Modified** — If `Explorer/Assets/Scenes/Main.unity` or its `.meta` appears in the changed files:
78150
> ⚠️ **Main scene modified** (`Explorer/Assets/Scenes/Main.unity`). This file is rarely changed intentionally — verify this wasn't pushed by mistake.
79151
80-
IMPORTANT: At the very end of your output, emit exactly these four lines (order matters — downstream automation parses them):
152+
--- STEP 9 — Verdict ---
153+
At the very end of your output, emit exactly these four lines (order matters — downstream automation parses them):
81154
REVIEW_RESULT: PASS ✅ (or FAIL ❌)
82155
COMPLEXITY: SIMPLE (or COMPLEX)
83156
COMPLEXITY_REASON: <one sentence citing which subsystem(s) the diff touches>

0 commit comments

Comments
 (0)