Skip to content

Commit b030de7

Browse files
sanbuphyclaude
andcommitted
Fix awkward translated terms in both Chinese and English lectures
Replace stiff English-to-Chinese calques with natural Chinese phrasing: - 自举契约 → 启动就绪清单 (bootstrap contract → startup readiness checklist) - 冷启动测试 → 全新会话测试 (cold start test → fresh session test) - 连续性工件 → 状态持久化文件 (continuity artifact → state persistence file) - 中间迷失效应 → 长文本中间信息容易被忽略 - 路由文件 → 入口文件 (router file → entry file) - 渐进式披露 → 按需展开 (progressive disclosure → reveal on demand) - And 8 more term fixes across lectures 01-06 Also: standardize summary section header to 核心要点 / Key Takeaways, add structured bullet points with bold labels to Lecture 01 failure modes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 8692fae commit b030de7

12 files changed

Lines changed: 90 additions & 94 deletions

File tree

  • docs
    • en/lectures
      • lecture-01-why-capable-agents-still-fail
      • lecture-02-what-a-harness-actually-is
      • lecture-03-why-the-repository-must-become-the-system-of-record
      • lecture-04-why-one-giant-instruction-file-fails
      • lecture-05-why-long-running-tasks-lose-continuity
      • lecture-06-why-initialization-needs-its-own-phase
    • zh/lectures
      • lecture-01-why-capable-agents-still-fail
      • lecture-02-what-a-harness-actually-is
      • lecture-03-why-the-repository-must-become-the-system-of-record
      • lecture-04-why-one-giant-instruction-file-fails
      • lecture-05-why-long-running-tasks-lose-continuity
      • lecture-06-why-initialization-needs-its-own-phase

docs/en/lectures/lecture-01-why-capable-agents-still-fail/index.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -19,17 +19,13 @@ OpenAI's 2025 harness engineering article puts it even more bluntly. They said t
1919

2020
## Where Agents Actually Get Stuck
2121

22-
The specific failure modes really come down to just a handful.
22+
The specific failure modes really come down to just a handful:
2323

24-
The first type: the task was never clearly defined in the first place. "Add a search feature" — that sentence means almost nothing. Search what? Full-text or structured queries? Should results be paginated? Highlighted? You didn't spell it out, so the agent has to guess. A correct guess is luck; a wrong one means rework that costs several times more than being specific would have in the first place.
25-
26-
Architectural conventions are another hidden pitfall. Your whole team uses the new SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must go through OAuth 2.0 authentication, but that rule only exists in your head and a Slack message from three months ago. The agent has no idea — it's not that it doesn't want to comply, it literally has never seen the rule.
27-
28-
The environment is a frequent source of trouble too. Incomplete dev setup, missing dependencies, wrong tool versions — the agent burns precious context window on `pip install` errors and Node version conflicts instead of doing the actual work you gave it.
29-
30-
Even more common is the complete absence of verification. No tests, no lint, or verification commands that were never communicated to the agent. The agent writes code, looks it over, decides it seems fine, and declares completion. Anthropic also observed an interesting phenomenon: when agents sense their context is running low, they rush to finish, skip verification steps, and choose a simple solution over the optimal one. They call this "context anxiety."
31-
32-
Tasks that span multiple sessions fare even worse. All discoveries from the previous session are lost. Every new session has to re-explore the project structure and re-understand the code organization. Agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.
24+
- **Vague requirements — the agent can only guess.** "Add a search feature" — that sentence means almost nothing. Search what? Full-text or structured queries? Should results be paginated? Highlighted? You didn't spell it out, so the agent has to guess. A correct guess is luck; a wrong one means rework that costs several times more than being specific would have in the first place.
25+
- **Implicit conventions not written down — the agent has no way to comply.** Your whole team uses the new SQLAlchemy 2.0 syntax, but the agent writes 1.x code by default. All API endpoints must go through OAuth 2.0 authentication, but that rule only exists in your head and a Slack message from three months ago. The agent has no idea — it's not that it doesn't want to comply, it literally has never seen the rule.
26+
- **Incomplete environment setup — the agent spends energy fixing the environment.** Incomplete dev setup, missing dependencies, wrong tool versions — the agent burns precious context window on `pip install` errors and Node version conflicts instead of doing the actual work you gave it.
27+
- **No verification methods — the agent calls it done when it feels done.** No tests, no lint, or verification commands that were never communicated to the agent. The agent writes code, looks it over, decides it seems fine, and declares completion. Anthropic also observed an interesting phenomenon: when agents sense their context is running low, they rush to finish, skip verification steps, and choose a simple solution over the optimal one. They call this "context anxiety."
28+
- **Cross-session state loss — every new session starts from scratch.** All discoveries from the previous session are lost. Every new session has to re-explore the project structure and re-understand the code organization. Agents without persistent state see failure rates spike sharply on tasks exceeding 30 minutes.
3329

3430
## Key Terminology
3531

docs/en/lectures/lecture-02-what-a-harness-actually-is/index.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Look at a few tools you already know:
3333
- **The repo is the single source of truth**: Anything the agent cannot see, for all practical purposes, does not exist. OpenAI treats the repo as the "system of record" — all necessary context must live there, delivered through structured files and clear directory organization.
3434
- **Give a map, not a manual**: OpenAI's experience is that `AGENTS.md` should be a directory page, not an encyclopedia. Around 100 lines is enough. If it does not fit, split it into a `docs/` directory and let the agent read on demand.
3535
- **Constrain, don't micromanage**: A good harness uses executable rules to constrain the agent, rather than enumerating instructions one by one. OpenAI says "enforce invariants, don't micromanage implementation"; Anthropic found that agents confidently praise their own work, and the solution is to separate "the person who does the work" from "the person who checks the work."
36-
- **Remove components one at a time**: To quantify each harness component's marginal contribution, remove them one at a time and see which removal causes the biggest performance drop. This tells you which components are most valuable right now, and it also reveals which ones are not yet contributing meaningfully. Anthropic used this method and discovered that as models get stronger, some components stop being critical — but new critical components always emerge.
36+
- **Remove one at a time and observe**: To quantify each harness component's marginal contribution, remove them one at a time and see which removal causes the biggest performance drop. This tells you which components are most valuable right now, and it also reveals which ones are not yet contributing meaningfully. Anthropic used this method and discovered that as models get stronger, some components stop being critical — but new critical components always emerge.
3737

3838
## The Five-Subsystem Harness Model
3939

@@ -68,7 +68,7 @@ Verification commands:
6868

6969
Missing any one of the five subsystems means an incomplete harness, and the agent will always feel awkward to use.
7070

71-
**Quantifying harness component value**: Use an "ablation experiment under the same model." Keep the model fixed, remove the five subsystems one at a time, and see which subsystem's removal causes the biggest performance drop. The component with the largest drop has the highest marginal contribution for the current task and is worth prioritizing. Whether to strengthen it depends on failure attribution, not just the size of the drop. Components with near-zero impact should not be dismissed outright: they may be redundant, poorly designed, or simply not exercised by the current task. This experiment answers "which component is most valuable right now" — it cannot, by itself, prove "where the bottleneck is." To truly locate a bottleneck, you must first examine failure records and attributions: was the task unclear, was context insufficient, was the environment unreproducible, was verification feedback missing, or was state management broken? Component ablation results can only serve as supporting evidence.
71+
**Quantifying harness component value**: Use a "controlled variable exclusion test." Keep the model fixed, remove the five subsystems one at a time, and see which subsystem's removal causes the biggest performance drop. The component with the largest drop has the highest marginal contribution for the current task and is worth prioritizing. Whether to strengthen it depends on failure attribution, not just the size of the drop. Components with near-zero impact should not be dismissed outright: they may be redundant, poorly designed, or simply not exercised by the current task. This experiment answers "which component is most valuable right now" — it cannot, by itself, prove "where the bottleneck is." To truly locate a bottleneck, you must first examine failure records and attributions: was the task unclear, was context insufficient, was the environment unreproducible, was verification feedback missing, or was state management broken? Component ablation results can only serve as supporting evidence.
7272

7373
## A Team's Real Story
7474

@@ -89,7 +89,7 @@ Four iterations, the model did not change at all, and success rate went from 20%
8989
- Harness = Instructions + Tools + Environment + State + Feedback. All five subsystems are essential.
9090
- If it is not model weights, it is harness. Your harness determines how much of the model's capability gets realized.
9191
- Among the five subsystems, the feedback subsystem usually has the lowest investment and highest return. Get your verification commands right first.
92-
- Use "ablation experiments under the same model" to quantify each subsystem's marginal contribution; to locate the real bottleneck, rely on failure records and attribution, not ablation alone.
92+
- Use "controlled variable exclusion tests" to quantify each subsystem's marginal contribution; to locate the real bottleneck, rely on failure records and attribution, not ablation alone.
9393
- Harness rots like code does. Audit regularly, and pay down harness debt just like you pay down technical debt.
9494

9595
## Further Reading
@@ -104,6 +104,6 @@ Four iterations, the model did not change at all, and success rate went from 20%
104104

105105
1. **Five-tuple harness audit**: Take a project where you currently use an AI agent and do a complete audit using the five-tuple framework. Score each subsystem 1-5. Find the lowest-scoring subsystem, spend 30 minutes improving it, and then observe the change in agent performance.
106106

107-
2. **Component-value ablation under the same model**: Pick one model and one challenging task. Sequentially remove instructions (delete AGENTS.md), remove feedback (do not provide verification commands), remove state (no progress files) — removing only one at a time, and measure the performance drop. Use the results to rank each subsystem's marginal value for the current task. If you want to find the bottleneck, you must also record failure logs and do root-cause attribution alongside the ablation.
107+
2. **Controlled variable exclusion test**: Pick one model and one challenging task. Sequentially remove instructions (delete AGENTS.md), remove feedback (do not provide verification commands), remove state (no progress files) — removing only one at a time, and measure the performance drop. Use the results to rank each subsystem's marginal value for the current task. If you want to find the bottleneck, you must also record failure logs and do root-cause attribution alongside the ablation.
108108

109109
3. **Affordance analysis**: Find a scenario in your project where the agent "wants to do something but can't" (e.g., knows it should use parameterized queries but does not know your project's ORM patterns). Analyze whether this is a Gulf of Execution (does not know how to operate) or a Gulf of Evaluation (does not know whether it did things right), then design a harness improvement to bridge the gap.

docs/en/lectures/lecture-03-why-the-repository-must-become-the-system-of-record/index.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ flowchart LR
3434
Warning["If a rule isn't in the repo,<br/>the agent can't see it"] --> Agent
3535
```
3636

37-
How do you test whether your map is good enough? Run a "cold-start test": open a brand-new agent session, give it only the repository contents, and see if it can answer five basic questions.
37+
How do you test whether your map is good enough? Run a "fresh session test": open a brand-new agent session, give it only the repository contents, and see if it can answer five basic questions.
3838

3939
```mermaid
4040
flowchart TB
@@ -57,7 +57,7 @@ If it can't answer, the map has blank spots. Where the map is blank, the agent h
5757

5858
- **Knowledge Visibility Gap**: The proportion of total project knowledge that's NOT in the repository. The bigger the gap, the higher the agent's failure rate. You can estimate it like this: count all the implicit knowledge about this project that lives in people's heads, then see how much of it actually made it into the repo. The difference is your visibility gap.
5959
- **System of Record**: The code repository as the authoritative source for project decisions, architecture constraints, execution state, and verification standards. The repo has the final say — nowhere else counts. If the information "this road is closed" only lives in Old Zhang's head, then every single time you have to ask Old Zhang. Write it in the repo, and nobody has to ask.
60-
- **Cold-Start Test**: The five questions from the previous section. How many the agent can answer is how complete your map is.
60+
- **Fresh Session Test**: The five questions from the previous section. How many the agent can answer is how complete your map is.
6161
- **Discovery Cost**: How much context budget the agent burns to find a single key piece of information in the repo. The more hidden the information, the higher the discovery cost, and the less budget remains for the actual task. Critical information should be placed where the agent sees it first — not buried ten directory levels deep.
6262
- **Knowledge Decay Rate**: The proportion of knowledge entries in the repo that become stale per unit of time. Documentation drifting out of sync with code is the biggest enemy — worse than no documentation at all is documentation that's out of date.
6363
- **ACID Analogy**: Applying database transaction principles (Atomicity, Consistency, Isolation, Durability) to agent state management. We'll expand on this below.
@@ -68,7 +68,7 @@ If it can't answer, the map has blank spots. Where the map is blank, the agent h
6868

6969
**Principle 2: Use a standardized entry file.** `AGENTS.md` (or `CLAUDE.md`) is the agent's "landing page." It doesn't need to contain all information, but it must let the agent quickly answer three questions: "What is this project," "How do I run it," and "How do I verify it." 50–100 lines is enough.
7070

71-
**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the cold-start test must have an answer. This is an ongoing balance to maintain — not too much, not too little, just enough.
71+
**Principle 3: Minimal but complete.** Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the agent's decision quality, that rule shouldn't exist. But every question from the fresh session test must have an answer. This is an ongoing balance to maintain — not too much, not too little, just enough.
7272

7373
**Principle 4: Update with code.** Bind knowledge updates to code changes. The simplest approach: put architecture docs in the corresponding module directory. When you modify code, you naturally notice the doc. After code changes, CI can remind you to check whether the docs need updating.
7474

@@ -110,12 +110,12 @@ The team executed a transformation:
110110
3. Created a centralized `CONSTRAINTS.md` using explicit "MUST / MUST NOT" language for hard constraints
111111
4. Added `PROGRESS.md` in each service directory tracking current work status
112112

113-
After transformation: the same agent could answer all key project questions on cold start, and task completion quality improved significantly.
113+
After transformation: the same agent could answer all key project questions on a fresh session, and task completion quality improved significantly.
114114

115115
## Key Takeaways
116116

117117
- Knowledge not in the repo doesn't exist for the agent. Putting critical decision information into the repository is the most fundamental harness investment — draw a good map so you don't get lost.
118-
- Use the "cold-start test" to evaluate repo quality: can a brand-new session answer five basic questions using only repo contents?
118+
- Use the "fresh session test" to evaluate repo quality: can a brand-new session answer five basic questions using only repo contents?
119119
- Knowledge should be near code, minimal but complete, and updated together with code. It's not about writing more docs — it's about putting information in the right place.
120120
- Use ACID principles for agent state: atomic commits, consistency verification, concurrency isolation, and durable critical knowledge.
121121
- Knowledge decay is the biggest enemy. Out-of-date documentation is more dangerous than no documentation at all — it sends the agent in the wrong direction while the agent thinks it's on the right track.
@@ -130,7 +130,7 @@ After transformation: the same agent could answer all key project questions on c
130130

131131
## Exercises
132132

133-
1. **Cold-start test**: Open a completely fresh agent session in your project (provide no verbal context whatsoever), let it see only the repository contents, then ask it five questions: What is this system? How is it organized? How do I run it? How do I verify it? What's the current progress? Record which ones it can't answer, then improve the repo until it can answer all of them.
133+
1. **Fresh session test**: Open a completely fresh agent session in your project (provide no verbal context whatsoever), let it see only the repository contents, then ask it five questions: What is this system? How is it organized? How do I run it? How do I verify it? What's the current progress? Record which ones it can't answer, then improve the repo until it can answer all of them.
134134

135135
2. **Knowledge externalization quantification**: List all decisions and constraints important to development work in your project. Mark each item as inside or outside the repo. Calculate your knowledge visibility gap (the proportion of items outside the repo). Make a plan to bring the gap below 10%.
136136

0 commit comments

Comments
 (0)