Skip to content

Commit bcd8319

Browse files
d-morrisonclaude
andauthored
docs: mine the four-month game-dev field report for supervision, tiering, and validation lessons (closes #98) (#184)
* docs: mine the four-month game-dev field report for supervision, tiering, and validation lessons (closes #98) Re-lands the change that PR #179 carried. That PR was stacked on the community-directory branch and merged into it, and the branch was then closed and re-landed as #182 without these commits, so the field-report edits never reached main. Same content as #179's reviewed head, plus one grammar fix Copilot raised on the stacked branch ("with coding agents and no coding experience"). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE * docs: scope the verification claim to the report and list the three cost levers Copilot's round on #184: the sentence "No automated verification appears anywhere in the project" asserted more than the report evidences, so it now says what the report and its thread do not mention; the three cost levers are a bullet list with the spend figure in its own sentence, per the lab's list rule; and the new callout uses the same div syntax as the one above it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE * docs: cite the field report by its post permalink The four citations linked the subreddit because the issue body carried no permalink; a comment on issue #98 does carry it, so the citations now link the post itself with the subreddit and date beside it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE * docs: one clause per line in the three field-report sentences Copilot flagged Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE * docs: one clause per line in the field-report citation sentence Copilot flagged the remaining long line in small-local-models-agentic.qmd. The split is source-only: no blank line is added, so the paragraph renders unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE * docs: caution callout and sentence-level lines for the field-report passages Copilot's suppressed comments: the worked example in responsibility-for-validation.qmd describes a failure mode, so it is a callout-caution rather than a callout-tip; the "budget for it" item in agents-best-practices.qmd now starts its continuation as a sentence and carries one clause per line. The citation parentheticals in both files are split at their separators to match the other two chapters. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GMS285msTfSxnF4Cy9RkTE --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 4a8e53d commit bcd8319

4 files changed

Lines changed: 70 additions & 1 deletion

File tree

chapters/ai-tools/agents-best-practices.qmd

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,14 @@ To work with coding agents safely and successfully:
1313
3. **Test thoroughly**:
1414
AI-generated code must be tested as rigorously as code you write yourself.
1515
Don't skip testing because "the AI wrote it."
16+
Budget for it, too.
17+
Two commenters on the same field report
18+
(["Vibe coded this game in four months"](https://www.reddit.com/r/ClaudeCode/comments/1vvhrfq/),
19+
r/ClaudeCode, 2026-08-22;
20+
summarized in [issue #98](https://github.com/Morrison-Lab/wai/issues/98))
21+
independently named manual testing as the dominant cost once generation is cheap.
22+
One put it as needing either deep domain experience or heavy testing
23+
to get a defect-free result.
1624

1725
4. **Start small**:
1826
Begin with small,

chapters/ai-tools/relative-advantages-ai-humans.qmd

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,29 @@ and AI agents will handle many of the computational and coding details.
4949
For most tasks,
5050
you won't need to step in and manipulate code yourself.
5151
However,
52-
you'll still need strong coding skills to:
52+
you'll still need enough domain judgment to:
5353

5454
- Supervise and validate AI-generated code
5555
- Handle edge cases that agents struggle with
5656
- Make creative decisions about architecture and design
5757
- Understand when agent suggestions are incorrect or suboptimal
5858

59+
Strong coding skills are one route to that judgment,
60+
and not the only one.
61+
A public field report from a non-programmer
62+
(["Vibe coded this game in four months"](https://www.reddit.com/r/ClaudeCode/comments/1vvhrfq/),
63+
r/ClaudeCode, 2026-08-22;
64+
summarized in [issue #98](https://github.com/Morrison-Lab/wai/issues/98))
65+
describes four months of building a browser racing game
66+
with coding agents and no coding experience at all,
67+
crediting knowledge of game technology and design,
68+
plus long practice with language models,
69+
for the planning and the early catches that kept the project on track;
70+
by the author's own account,
71+
lacking that knowledge as well would have stalled the project after a couple of prompts.
72+
The requirement is the supervision,
73+
whichever background supplies it.
74+
5975
#### Future Developments: World Models
6076

6177
As AI technology advances,

chapters/ai-tools/responsibility-for-validation.qmd

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,29 @@ Never blindly use AI-generated code without fully understanding it.
1616
If you don't completely understand what the AI has suggested,
1717
take the time to learn or ask a colleague for help.
1818
:::
19+
20+
::: callout-caution
21+
#### A worked example: the defects the builder cannot see
22+
23+
A public field report from a non-programmer
24+
(["Vibe coded this game in four months"](https://www.reddit.com/r/ClaudeCode/comments/1vvhrfq/),
25+
r/ClaudeCode, 2026-08-22;
26+
summarized in [issue #98](https://github.com/Morrison-Lab/wai/issues/98))
27+
describes four months of building a browser racing game with coding agents,
28+
and reports that most things worked on the first try.
29+
The comment thread under it says otherwise.
30+
Players reported:
31+
32+
- a minimap announcing a turn in the wrong direction
33+
- speed boosts invisible from any distance
34+
- an orbit track with no visible route
35+
- an accelerator with no consequences
36+
37+
Every one of those defects looks correct to the person who built it,
38+
because the builder reads the underlying state in their own head
39+
and sees the display agree with it.
40+
They surface only on contact with someone who has nothing but the display.
41+
Neither the report nor its thread mentions any automated verification.
42+
That is the answer to "why bother instrumenting checks when the thing looks fine":
43+
the thing looks fine to you precisely because you know what it is supposed to do.
44+
:::

chapters/ai-tools/small-local-models-agentic.qmd

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,25 @@ Two shapes of this pattern are worth knowing:
322322
wire a cloud-with-local-fallback endpoint,
323323
which composes with this split.
324324

325+
The same split is the main cost lever in a public field report
326+
from a non-programmer
327+
(["Vibe coded this game in four months"](https://www.reddit.com/r/ClaudeCode/comments/1vvhrfq/),
328+
r/ClaudeCode, 2026-08-22;
329+
summarized in [issue #98](https://github.com/Morrison-Lab/wai/issues/98)),
330+
who built a browser racing game over four months with coding agents.
331+
The report names three levers:
332+
333+
- plan with the strongest model available
334+
- implement with cheaper ones
335+
- keep the scope to what can realistically ship
336+
337+
The reported total was roughly $200
338+
in project-specific subscriptions over the four months,
339+
on top of a general-purpose subscription the author already held.
340+
The lab's machine-facing configuration,
341+
[`Morrison-Lab/ai-config`](https://github.com/Morrison-Lab/ai-config),
342+
states the same routing rule for agents.
343+
325344
None of these routing choices substitutes for the guardrails below.
326345
A well-chosen planner still hands off to an executor that can make a
327346
per-step mistake,

0 commit comments

Comments
 (0)