Skip to content

docs(tutorials): fix 2 cross-tutorial inconsistencies surfaced by a holistic audit - #27

Merged
wanshuiyin merged 1 commit into
mainfrom
fix/cross-tutorial-consistency
Jun 29, 2026
Merged

docs(tutorials): fix 2 cross-tutorial inconsistencies surfaced by a holistic audit#27
wanshuiyin merged 1 commit into
mainfrom
fix/cross-tutorial-consistency

Conversation

@wanshuiyin

Copy link
Copy Markdown
Owner

Holistic collection audit → 2 cross-tutorial fixes

A "look back" audit of the whole 28-tutorial collection — structural (inventory, counts, links, EN/ZH parity, gate, audit-chain) + a cross-tutorial consistency pass over shared concepts (FlashAttention, attention O(L²), KV cache/MLA, RoPE, Pre/Post-LN, μP, AdamW, MoE routing, GQA/MQA, quantization).

Result: the corpus is healthy and highly consistent — structurally all green (28× five-piece complete, 57 OK gate, EN/ZH parity 28/28), and 8/10 shared concepts fully aligned with 5/5 cross-references valid. Two reconcilable inconsistencies were found and are fixed here (both ZH + EN editions):

1. Warmup takeaway (normalization §5.2) — Medium

normalization_init_tutorial flatly stated "Pre-LN → warmup-free" (and "GPT/LLaMA are uniformly Pre-LN"), which reads as the opposite of optimizer_lr_schedule_tutorial §7.1 ("all schedules should be preceded by warmup", whose reason 1 — Adam early-step v̂ variance — is architecture-independent and applies to Pre-LN too). Real Pre-LN LLMs (GPT-3/LLaMA) do warm up.
Fix: a clarifying Note in §5.2 — Pre-LN removes only the gradient-imbalance reason for warmup; modern Pre-LN LLMs still warm up for Adam-variance / large-batch reasons (cross-refs optimizer §7.1). Both tutorials were individually correct; now they read consistently.

2. DeepSeek-V2 MLA compression ratio — Low

The same model appeared as three numbers across two tutorials:

  • kv_cache §4: 64× (counts only the bare latent d_c=512)
  • long_context §9.6 + table: 57× (the correct full cache d_c+d_r=576) ✓
  • long_context §9 intro: a loose ≈50×

Fix: aligned all to the full-cache ~57× (32768/576 ≈ 57), with the bare-latent 64× kept as an explicit caveat. The 57× already in long_context §9.6/§12.3 is the technically-correct figure.

Mechanics

Orchestrator-verified (arithmetic + warmup reasoning re-checked — this was a factual/clarity correction, not a new claim). The original gating reviews remain valid; the 6 HTMLs were re-rendered and the 6 sidecars' source_sha256 refreshed (+ a post_ship_review note on each documenting the fix).

Gate

python3 tools/verify_reviews.py --mode strict --reproduce57 OK · 0 FAIL · 3 EXEMPT · PASS.

🤖 Generated with Claude Code

…olistic audit

A holistic collection audit (structural + a cross-tutorial consistency pass over
shared concepts) found the corpus highly consistent (8/10 concepts fully aligned,
5/5 cross-references valid), with 2 reconcilable inconsistencies — fixed here
across both the ZH and EN editions:

1. Warmup takeaway (normalization §5.2): the tutorial flatly said "Pre-LN →
   warmup-free" (and "GPT/LLaMA are uniformly Pre-LN"), which reads as opposite to
   the optimizer tutorial's "all schedules should be preceded by warmup" (whose
   reason 1, Adam early-step v̂ variance, is architecture-independent and applies to
   Pre-LN too). Added a clarifying Note: Pre-LN removes only the *gradient-imbalance*
   reason for warmup; modern Pre-LN LLMs (GPT-3/LLaMA) still warm up for
   Adam-variance / large-batch reasons (see optimizer §7.1). Both tutorials were
   individually correct; now they read consistently.

2. DeepSeek-V2 MLA compression ratio: the same model appeared as 64× (kv_cache,
   counting only the bare latent d_c=512), 57× (long_context §9.6 + table, the
   correct full cache d_c+d_r=576), and a loose ≈50× (long_context §9 intro).
   Aligned all to the full-cache ~57× (32768/576≈57), with the bare-latent 64×
   noted as a caveat. The 57× figure already in long_context §9.6/§12.3 is the
   technically-correct one.

Orchestrator-verified (arithmetic + the warmup reasoning re-checked). The original
gating reviews remain valid; the 6 HTMLs were re-rendered and the 6 sidecars'
source hashes refreshed (+ a post_ship_review note documenting the fix).
Gate: tools/verify_reviews.py --mode strict --reproduce → 57 OK / PASS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wanshuiyin
wanshuiyin merged commit 81b7dcf into main Jun 29, 2026
1 check passed
@wanshuiyin
wanshuiyin deleted the fix/cross-tutorial-consistency branch June 29, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant