docs(tutorials): fix 2 cross-tutorial inconsistencies surfaced by a holistic audit - #27
Merged
Merged
Conversation
…olistic audit A holistic collection audit (structural + a cross-tutorial consistency pass over shared concepts) found the corpus highly consistent (8/10 concepts fully aligned, 5/5 cross-references valid), with 2 reconcilable inconsistencies — fixed here across both the ZH and EN editions: 1. Warmup takeaway (normalization §5.2): the tutorial flatly said "Pre-LN → warmup-free" (and "GPT/LLaMA are uniformly Pre-LN"), which reads as opposite to the optimizer tutorial's "all schedules should be preceded by warmup" (whose reason 1, Adam early-step v̂ variance, is architecture-independent and applies to Pre-LN too). Added a clarifying Note: Pre-LN removes only the *gradient-imbalance* reason for warmup; modern Pre-LN LLMs (GPT-3/LLaMA) still warm up for Adam-variance / large-batch reasons (see optimizer §7.1). Both tutorials were individually correct; now they read consistently. 2. DeepSeek-V2 MLA compression ratio: the same model appeared as 64× (kv_cache, counting only the bare latent d_c=512), 57× (long_context §9.6 + table, the correct full cache d_c+d_r=576), and a loose ≈50× (long_context §9 intro). Aligned all to the full-cache ~57× (32768/576≈57), with the bare-latent 64× noted as a caveat. The 57× figure already in long_context §9.6/§12.3 is the technically-correct one. Orchestrator-verified (arithmetic + the warmup reasoning re-checked). The original gating reviews remain valid; the 6 HTMLs were re-rendered and the 6 sidecars' source hashes refreshed (+ a post_ship_review note documenting the fix). Gate: tools/verify_reviews.py --mode strict --reproduce → 57 OK / PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Holistic collection audit → 2 cross-tutorial fixes
A "look back" audit of the whole 28-tutorial collection — structural (inventory, counts, links, EN/ZH parity, gate, audit-chain) + a cross-tutorial consistency pass over shared concepts (FlashAttention, attention O(L²), KV cache/MLA, RoPE, Pre/Post-LN, μP, AdamW, MoE routing, GQA/MQA, quantization).
Result: the corpus is healthy and highly consistent — structurally all green (28× five-piece complete, 57 OK gate, EN/ZH parity 28/28), and 8/10 shared concepts fully aligned with 5/5 cross-references valid. Two reconcilable inconsistencies were found and are fixed here (both ZH + EN editions):
1. Warmup takeaway (normalization §5.2) — Medium
normalization_init_tutorialflatly stated "Pre-LN → warmup-free" (and "GPT/LLaMA are uniformly Pre-LN"), which reads as the opposite ofoptimizer_lr_schedule_tutorial§7.1 ("all schedules should be preceded by warmup", whose reason 1 — Adam early-step v̂ variance — is architecture-independent and applies to Pre-LN too). Real Pre-LN LLMs (GPT-3/LLaMA) do warm up.Fix: a clarifying Note in §5.2 — Pre-LN removes only the gradient-imbalance reason for warmup; modern Pre-LN LLMs still warm up for Adam-variance / large-batch reasons (cross-refs optimizer §7.1). Both tutorials were individually correct; now they read consistently.
2. DeepSeek-V2 MLA compression ratio — Low
The same model appeared as three numbers across two tutorials:
kv_cache§4: 64× (counts only the bare latent d_c=512)long_context§9.6 + table: 57× (the correct full cache d_c+d_r=576) ✓long_context§9 intro: a loose ≈50×Fix: aligned all to the full-cache ~57× (32768/576 ≈ 57), with the bare-latent 64× kept as an explicit caveat. The 57× already in long_context §9.6/§12.3 is the technically-correct figure.
Mechanics
Orchestrator-verified (arithmetic + warmup reasoning re-checked — this was a factual/clarity correction, not a new claim). The original gating reviews remain valid; the 6 HTMLs were re-rendered and the 6 sidecars'
source_sha256refreshed (+ apost_ship_reviewnote on each documenting the fix).Gate
python3 tools/verify_reviews.py --mode strict --reproduce→ 57 OK · 0 FAIL · 3 EXEMPT · PASS.🤖 Generated with Claude Code