docs(tutorials): diffusion-cluster resweep — 66 fixes across 8 tutorials + What's New entry for the prior 20-file resweep

Second run of the full-collection resweep (see 59636aa / PR #31 for the first
20-file batch): the 8-file diffusion/generative-media cluster — 3d_generation,
diffusion_distillation, diffusion_foundations, diffusion_post_training,
flow_matching, image_generation_systems, vae_vqvae_vqgan, video_generation —
was the one part of the collection still on GPT-5.5-era review. As of this
PR, all 28 tutorials are under GPT-5.6-sol.

Same two-stage pipeline: GPT-5.6-sol found 67 candidate errors (fresh thread
per file) → an independent Claude pass adversarially re-verified each one
(try to refute before accepting) → 55 CONFIRMED + 11 NEEDS_NUANCE fixed, 2
REFUTED correctly left untouched. Notably higher P0 density than the first
batch — diffusion-model math has more subtle failure modes than the training/
systems/agent tutorials audited in round one.

Highlights:
- The single most-recurring error: conflating a "conditional interpolation
  path is a straight line" (true by construction, e.g. x_t=(1-t)x_0+tx_1)
  with "the marginal ODE trajectory the model learns is a straight line"
  (generally false under independent pairing — the marginal velocity field
  is a conditional expectation over all pairings sharing x_t, so it curves;
  this is exactly why Reflow exists). Recurred in diffusion_foundations
  (4 locations) and flow_matching (2 locations), each independently confirmed
  and fixed rather than resolved once and left inconsistent elsewhere.
- Real code bugs: iCT's Pseudo-Huber constant was missing the paper's √D
  scaling (understating it ~50x at CIFAR-10 scale); its noise schedule used
  i.i.d.-sampled-then-sorted lognormal σ instead of the paper's deterministic
  Karras/EDM grid; FSQ's bound function was missing the parity-offset/atanh
  shift needed for even quantization levels; LFQ's entropy-regularization
  loss maximized per-sample entropy (pushing every bit toward 0.5 — the
  opposite of confident quantization) instead of minimizing per-sample
  entropy while maximizing batch-marginal entropy; DDPO's pseudocode divided
  by a deterministic (zero-variance) terminal transition, producing NaN.
- Mechanism-level corrections: ADD's distillation loss was repeatedly
  described as teacher-output pixel MSE across 7+ locations in
  diffusion_distillation (it's a re-noise-then-single-teacher-denoise
  score-distillation signal — the file's own §4.1 already had this right,
  used as the alignment anchor); DMD2 was recommended as "GAN-free" when its
  core change from DMD v1 is adding a GAN loss; SD3's RF paper had its time
  direction backwards (t=0 is data, t=1 is noise, not the reverse); SD/SDXL's
  middle UNet block was said to have no cross-attention when it does
  (SpatialTransformer / UNetMidBlock2DCrossAttn); SDXL-Turbo's resolution was
  wrong (512px, not 1024px — that's SD3-Turbo); SVD's conditioning mechanism
  was misattributed (4+4=8 channel latent concat + CLIP cross-attn, not a
  mask-based 2C+1 scheme); Custom Diffusion was mischaracterized as a
  LoRA-DreamBooth variant when it directly optimizes cross-attention K/V
  (full-rank, no low-rank parametrization).
- 3 cross-file consistency flags caught in a final orchestrator pass: two
  fix agents correctly declined to touch content outside their assigned
  tutorial even when they spotted the same class of error recurring
  elsewhere (SD3-Turbo mislabeled as a Reflow example in diffusion_foundations
  when it's LADD; OpenMagViT2 cited as an FSQ usage example in
  vae_vqvae_vqgan when it's LFQ per that file's own table) — swept up here
  before merge rather than left as latent inconsistencies.

Mechanics: every touched .md got its EN edition fixed in parallel, both
HTMLs re-rendered, both sidecars updated with an appended audit-round entry
(old rounds preserved) and a fresh source_sha256. flow_matching's standalone
runnable script was checked and needed no changes (independently re-verified
via direct math re-derivation, not just trusted from the audit JSON). Also
adds the README What's New entry (both editions) for the prior 20-file
resweep (PR #31 / 59636aa), which had landed without one.

Gate: tools/verify_reviews.py --mode strict --reproduce → 59 OK / 0 WARN /
0 FAIL / 3 EXEMPT / PASS.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: wanshuiyin

README.md

@@ -66,6 +66,7 @@ Phone on the subway, iPad at a café, laptop in the library — same HTML link o
 
 ## 📢 What's New
 
+- **2026-07-12** — ![QUALITY](https://img.shields.io/badge/QUALITY-2ea44f?style=flat-square) 🔍 **Full-collection resweep — 20 tutorials, 236 fixes, cross-model review upgraded to GPT-5.6-sol** — after the reviewer model moved from GPT-5.5 to GPT-5.6-sol, a full re-audit of 20 tutorials (training fundamentals / attention·long-context / RLHF·KL·OPD / inference systems / PEFT / reasoning·agents / multi-agent·self-evolving / RAG·VLM; the 8-file diffusion/generative-media cluster is deferred, still under GPT-5.5-era review). **Two-stage pipeline**: GPT-5.6-sol scanned every file for errors (240 candidate findings) → an independent Claude pass adversarially re-verified each one (try to refute before accepting) → 191 CONFIRMED + 45 NEEDS_NUANCE fixed, 4 REFUTED correctly left untouched. Fixed 5 **real code bugs** (an E4M3FN encoder round-trip saturation bug, `RowParallelLinear` using the local shard's fan_in instead of global, `train_step_4d` crashing on an empty list, GRPO's length-biased KL-loss normalization, `DoRALinear` missing device/dtype propagation) plus a long list of mechanism-level misconceptions (DeepSeek-V3's routing formula, MLA absorb's compute/bandwidth trade-off, ZeRO++ hpZ/qgZ, Mixtral's parameter accounting now matching the official 46.7B/12.9B exactly, BLIP-2/Qwen-VL/LLaVA-NeXT connector architectures, and more). **StarPO's acronym** (RAGEN, arXiv 2504.20073) — two independent verification rounds gave opposite answers, resolved by fetching the paper's abstract directly: State-Thinking-Actions-Reward Policy Optimization. All 20 files' ZH+EN editions fixed in parallel, re-rendered, sidecars updated. Gate: `tools/verify_reviews.py --mode strict --reproduce` → 59 OK / 0 FAIL / PASS. ([#31](https://github.com/wanshuiyin/ARIS-in-AI-Offer/pull/31) · [59636aa](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/59636aa))
 - **2026-07-11** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🔤 **#29 Tokenization cheat sheet** — the foundation the whole collection had been taking for granted: the **pipeline mental model** (chat template / special-token recognition upstream · normalization → pre-tokenization → segmentation model → IDs → post-processing) · **BPE / WordPiece / Unigram** (training ≠ encoding — the #1 hand-coding trap) · **byte-level BPE vs byte fallback** (`Ġ` vs `▁`) · **the five vocab quantities** (mergeable · special · len(tokenizer) · max-ID+1 · embedding rows — cl100k has ID holes, Qwen pads by checkpoint size) · **fertility / BPB metrics** · **vocab extension** (lexical vs control paths) · **production traps** (prefix instability / token healing · special-token injection · glitch tokens · offset mapping) · **tokenizer-free** (ByT5 / MegaByte / BLT). **Bilingual** (中文 + EN), with a runnable script ([`code/tokenization.py`](docs/tutorials/code/tokenization.py) — exact GPT-2 `bytes_to_unicode`, frozen-rank BPE encoding, both Unigram DPs (forward 0.22350625 hand-checked), vocab-extension regression; pure stdlib, verified on a real box) and 25 高频题. First tutorial with cross-model review moved UP to the design stage (81 GPT-5.6-sol guardrails before drafting + 5 review batches + 1 dispute round + an ultra final SHIP — corrections flowed both ways: the reviewer caught 6 hard errors, the orchestrator overturned the reviewer's Qwen vocab claim). [`tokenization_tutorial.html`](docs/tutorials/tokenization_tutorial.html).
 - **2026-06-29** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) ⚙️ **#28 Optimizers & LR Schedules cheat sheet** — the optimizer/schedule training-mechanics gap: **SGD·Momentum·Nesterov** · **AdaGrad→RMSProp→Adam** (the adaptive lineage) · **Adam bias correction** (the uncorrected first step is ~3.16× too *large*, not small — the direction trap) · **AdamW decoupled weight decay** (Adam's L2 ≠ weight decay) · **frontier** (Muon / Lion / Shampoo / SOAP / Adafactor / LAMB / Sophia) · **LR schedules** (warmup / cosine / Noam / WSD / one-cycle) · weight decay + LR-batch scaling + no-decay groups · grad clipping + LLM hyperparams (β₂=0.95). **Bilingual** (中文 + EN), with a runnable script ([`code/optimizer_lr_schedule.py`](docs/tutorials/code/optimizer_lr_schedule.py) — SGD/Adam/AdamW from-scratch vs `torch.optim`, the executable **AdamW≠Adam+L2** proof, the bias-correction direction, cosine-warmup, verified on a real box) and 25 高频题. [`optimizer_lr_schedule_tutorial.html`](docs/tutorials/optimizer_lr_schedule_tutorial.html).
 - **2026-06-19** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧱 **#27 Normalization / Residual / Init cheat sheet** — the foundational training-mechanics hole: **BatchNorm / LayerNorm / RMSNorm / GroupNorm** · **Pre-vs-Post-LN** (Xiong's gradient argument — why Post-LN needs warmup) · **DeepNorm / Sandwich / QK-Norm** · **residual connections + scaling** (LayerScale / ReZero / GPT-2's 1/√(2N)) · **Xavier vs Kaiming** (the ReLU factor-2, and why the preserved quantity is the second moment E[y²], not Var) · **μP** (width-invariant HP transfer) · **Fixup / NFNets / DyT** (norm-free) · plus the **covariate-shift debunk** (Santurkar). **Bilingual** (中文 + EN), with a runnable script ([`code/normalization.py`](docs/tutorials/code/normalization.py) — from-scratch LN/RMSNorm vs `torch` + Pre/Post-LN gradient + Kaiming second-moment + GPT-2 residual scaling, verified on a real box) and 25 高频题. The cross-model review caught a real methodological confound in the Pre/Post-LN gradient demo (an output-normalization artifact) and redesigned it to a loss-robust top/bottom weight-grad ratio. [`normalization_init_tutorial.html`](docs/tutorials/normalization_init_tutorial.html).

README_CN.md

@@ -66,6 +66,7 @@
 
 ## 📢 最新动态
 
+- **2026-07-12** — ![QUALITY](https://img.shields.io/badge/QUALITY-2ea44f?style=flat-square) 🔍 **全集重扫 —— 20 篇教程、236 处修复，跨模型审升级到 GPT-5.6-sol** —— 审稿模型从 GPT-5.5 升级到 GPT-5.6-sol 后，对训练基础 / attention·长上下文 / RLHF·KL·OPD / 推理系统 / PEFT / reasoning·agent / 多智能体·自演化 / RAG·VLM 共 20 篇（扩散生成簇 8 篇暂缓，仍在 5.5 时代审查）做了一轮完整重审。**两阶段流水线**：GPT-5.6-sol 逐篇找错（240 条候选）→ 独立 Claude 对每条做对抗验证（尝试先推翻再接受）→ 191 CONFIRMED + 45 NEEDS_NUANCE 落地修复，4 条 REFUTED 正确保留原文。修了 5 处**真实代码 bug**（E4M3FN 编码器 round-trip 错误饱和、`RowParallelLinear` 用错本地 fan_in、`train_step_4d` 空列表崩溃、GRPO KL loss 长度偏置归一化、`DoRALinear` 缺 device/dtype）和大量机制级误解（DeepSeek-V3 路由公式、MLA absorb 的算力/带宽权衡、ZeRO++ hpZ/qgZ、Mixtral 参数拆账精确复现官方 46.7B/12.9B、BLIP-2/Qwen-VL/LLaVA-NeXT 连接器架构等）。**StarPO（RAGEN, arXiv 2504.20073）缩写**两轮验证给出相反结论后，直接查了论文摘要原文定案为 State-Thinking-Actions-Reward Policy Optimization。全部 20 篇 ZH+EN 同步、重渲染、sidecar 留痕。Gate：`tools/verify_reviews.py --mode strict --reproduce` → 59 OK / 0 FAIL / PASS。（[#31](https://github.com/wanshuiyin/ARIS-in-AI-Offer/pull/31) · [59636aa](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/59636aa)）
 - **2026-07-11** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🔤 **第 29 篇 Tokenization / 分词器 cheat sheet 上线** —— 补分词这个全集一直默认已知的地基：**pipeline 心智模型**（chat template/special 识别在上游 · normalization → pre-tokenization → 分词模型 → ID → post-processing）· **BPE / WordPiece / Unigram**（训练 ≠ 编码——最高频手写陷阱）· **byte-level BPE vs byte fallback**（`Ġ` vs `▁`）· **词表五量**（mergeable · special · len(tokenizer) · max-ID+1 · embedding 行——cl100k 有 ID 空洞、Qwen 按尺寸 padding）· **fertility / BPB 口径** · **扩词表**（lexical vs control 两条路）· **生产陷阱**（prefix instability / token healing · special 注入 · glitch token · offset mapping）· **tokenizer-free**（ByT5 / MegaByte / BLT）。**双语**（中文 + EN），含可跑脚本（[`code/tokenization.py`](docs/tutorials/code/tokenization.py) —— 精确复现 GPT-2 `bytes_to_unicode`、BPE 冻结 rank 编码、Unigram 双 DP（forward 0.22350625 手算核对）、扩词表 regression，纯标准库真机验证）和 25 高频题。本篇首次把跨模型审前置到设计阶段（GPT-5.6-sol 起草前给 81 条护栏 + 5 批审 + 1 轮对线 + ultra 终审 SHIP——双向纠错：审稿人抓 6 处硬错，orchestrator 反翻其 Qwen 词表口径一条）。[`tokenization_tutorial.html`](docs/tutorials/tokenization_tutorial.html)。
 - **2026-06-29** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) ⚙️ **第 28 篇 优化器 & LR Schedule cheat sheet 上线** —— 补优化器 / 调度训练机理这个洞：**SGD·Momentum·Nesterov** · **AdaGrad→RMSProp→Adam**（自适应谱系）· **Adam 偏差修正**（不修正首步是 ~3.16× **偏大**不是偏小——方向陷阱）· **AdamW 解耦权重衰减**（Adam 的 L2 ≠ weight decay）· **前沿**（Muon / Lion / Shampoo / SOAP / Adafactor / LAMB / Sophia）· **LR 调度**（warmup / cosine / Noam / WSD / one-cycle）· 权重衰减 + LR-batch 缩放 + no-decay 组 · 梯度裁剪 + LLM 超参（β₂=0.95）。**双语**（中文 + EN），含可跑脚本（[`code/optimizer_lr_schedule.py`](docs/tutorials/code/optimizer_lr_schedule.py) —— SGD/Adam/AdamW from-scratch vs `torch.optim`、可执行的 **AdamW≠Adam+L2** 证明、bias-correction 方向、cosine-warmup，真机验证）和 25 高频题。[`optimizer_lr_schedule_tutorial.html`](docs/tutorials/optimizer_lr_schedule_tutorial.html)。
 - **2026-06-19** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧱 **第 27 篇 归一化 / 残差 / 初始化 cheat sheet 上线** —— 补 foundational 训练机理这个洞：**BatchNorm / LayerNorm / RMSNorm / GroupNorm** · **Pre-vs-Post-LN**（Xiong 梯度论证——Post-LN 为何需 warmup）· **DeepNorm / Sandwich / QK-Norm** · **残差连接 + 缩放**（LayerScale / ReZero / GPT-2 的 1/√(2N)）· **Xavier vs Kaiming**（ReLU 的因子 2，且守恒的量是二阶矩 E[y²] 而非 Var）· **μP**（宽度不变超参迁移）· **Fixup / NFNets / DyT**（归一化-free）· 以及 **covariate-shift 辨析**（Santurkar）。**双语**（中文 + EN），含可跑脚本（[`code/normalization.py`](docs/tutorials/code/normalization.py) —— from-scratch LN/RMSNorm vs `torch` + Pre/Post-LN 梯度 + Kaiming 二阶矩 + GPT-2 残差缩放，真机验证）和 25 高频题。跨模型审抓到 Pre/Post-LN 梯度 demo 的一个真方法学混淆（输出归一化假象），改用对 loss 稳健的顶/底梯度比。[`normalization_init_tutorial.html`](docs/tutorials/normalization_init_tutorial.html)。