docs: mine the four-month game-dev field report for supervision, tiering, and validation lessons (closes #98)

[d-morrison]

Closes #98

Re-lands the change #179 carried. That PR was stacked on the community-directory branch and was merged into it, and the branch was then closed and re-landed as #182 without these commits, so the field-report edits never reached main. This PR carries the same four-file change as #179's reviewed head (60665c0), rebased onto current main as one commit, plus one grammar fix Copilot raised on the stacked branch ("with coding agents and no coding experience").

What the change does, condensed from the field report summarized in #98:

• relative-advantages-ai-humans.qmd: reframes "strong coding skills" as domain judgment, with the report's example
• small-local-models-agentic.qmd: adds the planner-versus-executor split as a cost lever, with the report's spend figure and its qualifier
• agents-best-practices.qmd: adds the two commenters' point that manual testing dominated the cost
• responsibility-for-validation.qmd: adds a callout on the four defects the builder could not see

🤖 Generated with Claude Code

https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE

---

Generated by Claude Code

[Copilot]

🟡 Changes recommended

A couple of the new documentation passages would benefit from clearer list formatting and more carefully scoped wording to avoid overstating what the cited field report actually evidences.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR re-lands the previously reviewed field-report-derived edits (from stacked PR #179) into main, adding sourced lessons on supervision (domain judgment), cost tiering (planner vs executor model choice), manual testing cost, and validation blind spots, as discussed in issue #98.

Changes:

• Reframes “strong coding skills” as “enough domain judgment” for supervising and validating agent output, with an attributed field-report example.
• Adds a planner/strong-model vs executor/cheaper-model routing takeaway (plus a reported spend figure) to the routed-architectures discussion.
• Adds two field-report comment-thread takeaways: manual testing as dominant cost, and a concrete validation cautionary example of defects visible to users but not the builder.

File summaries

File	Description
chapters/ai-tools/relative-advantages-ai-humans.qmd	Rewords supervision requirements toward domain judgment and adds an attributed field-report datapoint.
chapters/ai-tools/small-local-models-agentic.qmd	Adds a field-report example tying planner/executor model tiering to cost control.
chapters/ai-tools/agents-best-practices.qmd	Extends “Test thoroughly” with an attributed note to budget for manual testing effort.
chapters/ai-tools/responsibility-for-validation.qmd	Adds a callout illustrating validation failures that the builder can miss but users surface.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[d-morrison]

Round 1 dispositions, pushed as 070ebb7:

• Addressed (Copilot): the verification sentence is scoped to what the report and its thread say; the three cost levers are a bullet list with the spend figure in its own sentence.
• Addressed (Claude, non-blocking): the new callout uses the same ::: callout-tip div syntax as the callout-warning above it.
• Rebutted (Claude, non-blocking): the citations link the subreddit rather than the post because issue Mine a four-month AI-assisted game-dev field report: refine the division-of-labour framing, add model tiering, add a validation cautionary case #98 supplies no permalink and this repo's guidance forbids inventing URLs; the post title and date in the citation identify it.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[Copilot]

🟡 Changes recommended

Multiple new citations link to the subreddit instead of the specific field-report post, reducing traceability and verifiability of the added claims.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

chapters/ai-tools/agents-best-practices.qmd:18

• The field-report citation links to the r/ClaudeCode subreddit rather than the specific post, which makes the supporting evidence difficult to find. Use the post permalink (referenced in issue #98).
  chapters/ai-tools/relative-advantages-ai-humans.qmd:62
• This reference links to the r/ClaudeCode subreddit home, not the specific field-report thread. Linking the post permalink (referenced in issue #98) will make the attribution verifiable and durable.

• Files reviewed: 4/4 changed files
• Comments generated: 2
• Review effort level: Lite

[d-morrison]

Round 2 disposition, pushed as feb79b3:

• Addressed (Copilot, four citations): the field report is now cited by its post permalink, taken from a comment on issue Mine a four-month AI-assisted game-dev field report: refine the division-of-labour framing, add model tiering, add a validation cautionary case #98. This retracts the "Rebutted" line in my round-1 comment, which said no permalink was available; I had checked the issue body and not its comments.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[Copilot]

🟢 Approval recommended

The changes are documentation-only and consistent with the cited source/issue context, with only minor formatting nits (semantic line-break reflow) noted.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

chapters/ai-tools/responsibility-for-validation.qmd:36

• For consistency with the fragment’s semantic line breaks, split this sentence so each clause sits on its own line.
  chapters/ai-tools/small-local-models-agentic.qmd:335
• To match the fragment’s semantic line-break style (one clause per line), reflow this sentence so it doesn’t carry multiple clauses on a single long line.

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

[d-morrison]

Round 3 disposition, pushed as 79c21f5:

• Addressed (Copilot, three line-break nits): the three long sentences are reflowed one clause per line.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[Copilot]

🟡 Changes recommended

A few small documentation-quality issues (callout type signaling and line/grammar consistency) should be corrected to keep the guidance clear and consistent.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

chapters/ai-tools/agents-best-practices.qmd:20

• Within this list item, the continuation sentence after the colon should be capitalized for consistency with the other numbered items, and splitting the last sentence avoids a long run-on line.
  chapters/ai-tools/responsibility-for-validation.qmd:20
• This callout is describing a cautionary validation failure mode, so callout-tip may mis-signal the tone/importance to readers. Consider using a caution/warning callout type here.

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

[d-morrison]

Round 5 (head 003439e): Copilot's remaining thread on small-local-models-agentic.qmd (one long line in the field-report sentence) is Addressed by a source-only split, one clause per line, with no rendered change. Claude's round-4 verdict on 79c21f5 was clean with no findings; this push carries nothing else.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[d-morrison]

Round 6 (head 2a18463): Copilot's two suppressed "previously missed" comments are Addressed.

• responsibility-for-validation.qmd: the worked example describes a failure mode, so its callout is now callout-caution rather than callout-tip.
• agents-best-practices.qmd: the "Budget for it" item now ends its sentence at the colon, capitalizes the continuation, and splits the run-on line one clause per line.

The citation parenthetical is also split at its separators in those two files and in relative-advantages-ai-humans.qmd, matching the split already made in small-local-models-agentic.qmd, so all four copies read the same. No rendered content changed except the callout type.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[Copilot]

🟢 Approval recommended

The documentation updates are internally consistent, appropriately caveated, and align with existing formatting/linking conventions in the chapter.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Lite

[github-actions]

Claude finished review — View run

---

This matches the range summary exactly (20 insertions, 10 deletions, 4 files), and the commit messages confirm this round is responding to Copilot's remaining line-length and callout-semantics comments — nothing outside that scope was touched.

Review

This round's diff is exactly what the commit messages describe: chapters/ai-tools/responsibility-for-validation.qmd:20 changes ::: callout-tip to ::: callout-caution (a semantic fix — the block describes a failure mode uncaught by the builder, which is what callout-caution is for; callout-caution is already used elsewhere in this repo, e.g. chapters/ai-tools/running-agents-offline.qmd:31,288,538, so this is consistent with existing usage, not a new pattern). The remaining edits across all four files are pure source-line-wrap changes to the citation parenthetical and the "budget for it" sentence — no blank lines added/removed, no wording, links, or citation content changed.

Verification performed:

• Confirmed via git diff 79c21f58 acd49566 that only these 4 files changed, matching the range summary's stat (20 insertions, 10 deletions).
• Checked all four touched files for the bullet-list blank-line rule and div-fence balance — all intact (responsibility-for-validation.qmd:29-32, small-local-models-agentic.qmd:330-334, relative-advantages-ai-humans.qmd:52-55).
• ::: callout-caution (unbraced shorthand) matches the sibling ::: callout-warning immediately above it in the same file (responsibility-for-validation.qmd:14), so the div syntax stays internally consistent even though it differs from the :::: {.callout-caution} brace form used in running-agents-offline.qmd — that stylistic split predates this PR and isn't something this diff introduces.
• The Reddit citation URL, issue Mine a four-month AI-assisted game-dev field report: refine the division-of-labour framing, add model tiering, add a validation cautionary case #98 cross-reference, and Morrison-Lab/ai-config link are unchanged from prior rounds, which already verified them (issue Mine a four-month AI-assisted game-dev field report: refine the division-of-labour framing, add model tiering, add a validation cautionary case #98 content match, and ai-config is the lab's real machine-facing config repo referenced elsewhere in this same chapter).

No new issues found. The prior verdict (2026-09-01T23:25:27Z, #184 (comment) — "Claude finished review") still stands: this is a clean, well-scoped documentation PR, and this round's changes are a targeted response to Copilot's two remaining suppressed comments (callout semantics + one more long line), with zero effect on rendered content beyond the intentional callout-type change.

Verdict

Ready for merge

Structured Review Data (JSON)

{
  "schema_version": "1.0",
  "reviewer": "claude",
  "commit_sha": "acd495668c215927a648c04b05b68ff142be79ff",
  "verdict": "CLEAN",
  "findings": []
}

Reviewed commit: 2a18463

[github-actions]

💰 Cost: $0.3676 (review) — run

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-01 23:49 UTC