docs(tutorials): add Normalization / Residual / Init cheat sheet (#27, bilingual) + runnable normalization.py (#25)

#27 fills the foundational training-mechanics gap the collection never derived:
- BatchNorm / LayerNorm / RMSNorm / GroupNorm (dims, train≠eval, why LN for sequences)
- Pre-LN vs Post-LN (Xiong's gradient argument: Post-LN top-heavy Θ(d√ln d) → needs warmup)
- DeepNorm / Sandwich / QK-Norm · residual connections + scaling (LayerScale / ReZero / GPT-2 1/√(2N))
- Xavier vs Kaiming (the ReLU factor-2; preserved quantity is the SECOND MOMENT E[y²], not Var)
- μP (width-invariant HP transfer) · Fixup / NFNets / DyT (norm-free) · covariate-shift debunk (Santurkar)

Bilingual (中文 + EN), 25 L1/L2/L3 questions, runnable code/normalization.py
(from-scratch LN/RMSNorm vs torch + Pre/Post-LN gradient + Kaiming second-moment + GPT-2
residual scaling, verified on a real box; §A shows real output).

Cross-model reviewed on Codex GPT-5.5 xhigh: math + 2 citation batches + code + answers
+ overall + EN translation-fidelity. The code batch caught a real methodological confound in
the Pre/Post-LN gradient demo (a Post-LN output-normalization artifact made ||dL/dx|| look
like a 4e6× gap) — redesigned to the loss-robust per-block weight-grad top/bottom ratio
(Pre-LN 0.40 balanced vs Post-LN 2.35 top-heavy). Also corrected Kaiming's preserved quantity
to E[y²] (Var would be 1−1/π≈0.68). Gate: tools/verify_reviews.py --mode strict --reproduce → 55 OK / PASS.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: wanshuiyin

README.md

@@ -8,7 +8,7 @@
 >
 > 📖 **中文版 (Chinese version)**: [README_CN.md](README_CN.md)
 
-**📚 Jump to a topic** — 26 first-party cheat sheets across 7 categories + 1 community-contributed category:
+**📚 Jump to a topic** — 27 first-party cheat sheets across 7 categories + 1 community-contributed category:
 
 [🧠 General / Foundations](#-general--foundations) · [🎯 Post-Training & Reasoning](#-post-training--reasoning) · [🏛️ LLM Architecture & Systems](#-llm-architecture--systems) · [🌊 Generative Models — Theory & Tokenizers](#-generative-models--theory--tokenizers) · [🎨 Generation Systems (Image / Video / 3D / Diffusion Post-Training)](#-generation-systems--image--video--3d--diffusion-post-training) · [👁️ Multimodal](#-multimodal) · [🤖 Agents](#-agents) · [🦾 Embodied AI / 具身智能](#-embodied-ai--具身智能)
 
@@ -66,6 +66,7 @@ Phone on the subway, iPad at a café, laptop in the library — same HTML link o
 
 ## 📢 What's New
 
+- **2026-06-19** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧱 **#27 Normalization / Residual / Init cheat sheet** — the foundational training-mechanics hole: **BatchNorm / LayerNorm / RMSNorm / GroupNorm** · **Pre-vs-Post-LN** (Xiong's gradient argument — why Post-LN needs warmup) · **DeepNorm / Sandwich / QK-Norm** · **residual connections + scaling** (LayerScale / ReZero / GPT-2's 1/√(2N)) · **Xavier vs Kaiming** (the ReLU factor-2, and why the preserved quantity is the second moment E[y²], not Var) · **μP** (width-invariant HP transfer) · **Fixup / NFNets / DyT** (norm-free) · plus the **covariate-shift debunk** (Santurkar). **Bilingual** (中文 + EN), with a runnable script ([`code/normalization.py`](docs/tutorials/code/normalization.py) — from-scratch LN/RMSNorm vs `torch` + Pre/Post-LN gradient + Kaiming second-moment + GPT-2 residual scaling, verified on a real box) and 25 高频题. The cross-model review caught a real methodological confound in the Pre/Post-LN gradient demo (an output-normalization artifact) and redesigned it to a loss-robust top/bottom weight-grad ratio. [`normalization_init_tutorial.html`](docs/tutorials/normalization_init_tutorial.html).
 - **2026-06-18** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) ⚡ **#26 Linear / Sparse Attention cheat sheet** — the sub-quadratic / efficient-attention hole the collection kept referencing but never derived: **linear attention** (kernel φ + associativity → matrix-state RNN) · **SSM / Mamba** (selective S6) · **Mamba-2 / SSD** (1-semiseparable duality ≡ structured masked linear attention) · **DeltaNet / Gated DeltaNet** (overwrite update) · **chunkwise-parallel** training · **trainable sparse** (NSA three-branch / MoBA / Lightning / DSA) · **hybrid** (Jamba / Hymba / Qwen3-Next / Kimi-Linear / MiniMax-01). **Bilingual** (中文 + EN), with a runnable script ([`code/linear_sparse_attention.py`](docs/tutorials/code/linear_sparse_attention.py) — `chunkwise ≡ recurrent` equivalence + delta-rule + block-sparse, verified on a real box) and 25 高频题, settled through multi-batch Codex GPT-5.5 xhigh citation / math / code / answer / overall + render-fidelity review. [`linear_sparse_attention_tutorial.html`](docs/tutorials/linear_sparse_attention_tutorial.html) ([#22](https://github.com/wanshuiyin/ARIS-in-AI-Offer/pull/22) · [b84e913](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/b84e913)).
 - **2026-06-13 → 06-14** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧩 **Two must-know cheat sheets shipped: #24 LoRA / PEFT + #25 RAG + Embedding / Retrieval** — the two glaring holes the collection used everywhere but never derived. **LoRA/PEFT**: B=0 identity start · α/r vs rsLoRA √r · zero-latency merge · QLoRA (NF4 / double-quant / paged) · DoRA · the family vs Adapter / Prefix / Prompt / BitFit. **RAG**: bi-encoder · InfoNCE + hard negatives · Matryoshka · BM25 · HNSW · RRF hybrid · cross-encoder vs ColBERT late interaction · HyDE / Self-RAG / CRAG · GraphRAG · RAGAS. Both **bilingual** (中文 + EN), each with from-scratch PyTorch, a runnable script ([`code/lora.py`](docs/tutorials/code/lora.py) · [`code/rag_embedding.py`](docs/tutorials/code/rag_embedding.py), verified on a real box) and 25 高频题, settled through multi-round Codex GPT-5.5 xhigh math/code + render + EN translation-fidelity review. [`lora_peft_tutorial.html`](docs/tutorials/lora_peft_tutorial.html) · [`rag_embedding_retrieval_tutorial.html`](docs/tutorials/rag_embedding_retrieval_tutorial.html).
 - **2026-06-08** — ![POLISH](https://img.shields.io/badge/POLISH-blue?style=flat-square) 🔧 [`tools/render_html.py`](tools/render_html.py) now strips a leading UTF-8 BOM before frontmatter detection ([6cc4876](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/6cc4876)).
@@ -109,6 +110,7 @@ Long-form technical blogs — hand-authored, cross-model reviewed; outside the a
 | Topic | HTML 中文 | HTML EN | MD |
 |---|---|---|---|
 | **Attention Interview Cheat Sheet** | [📄 CN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/attention_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/attention_tutorial_en.html) | [MD](docs/tutorials/attention_tutorial.md) |
+| **Normalization / Residual / Init (BatchNorm / LayerNorm / RMSNorm / Pre-vs-Post-LN / DeepNorm / QK-Norm / Xavier·Kaiming / μP)** | [📄 CN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/normalization_init_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/normalization_init_tutorial_en.html) | [MD](docs/tutorials/normalization_init_tutorial.md) |
 | **KL Divergence in RLHF (k1/k2/k3 · placement gradient bias)** | [📄 CN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/kl_divergence_rlhf_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/kl_divergence_rlhf_tutorial_en.html) | [MD](docs/tutorials/kl_divergence_rlhf_tutorial.md) |
 
 ### 🎯 Post-Training & Reasoning

README_CN.md

@@ -8,7 +8,7 @@
 >
 > 📖 **English version (default)**: [README.md](README.md)
 
-**📚 按方向直达** —— 26 篇 first-party cheat sheet，7 大方向 + 1 个社区贡献方向：
+**📚 按方向直达** —— 27 篇 first-party cheat sheet，7 大方向 + 1 个社区贡献方向：
 
 [🧠 General / 基础](#-general--基础) · [🎯 Post-Training & Reasoning](#-post-training--reasoning) · [🏛️ LLM Architecture & Systems](#-llm-architecture--systems) · [🌊 Generative Models — 理论 & Tokenizers](#-generative-models--理论--tokenizers) · [🎨 Generation Systems（图像 / 视频 / 3D / Diffusion 后训练）](#-generation-systems--图像--视频--3d--diffusion-后训练) · [👁️ Multimodal](#-multimodal) · [🤖 Agents](#-agents) · [🦾 Embodied AI / 具身智能](#-embodied-ai--具身智能)
 
@@ -66,6 +66,7 @@
 
 ## 📢 最新动态
 
+- **2026-06-19** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧱 **第 27 篇 归一化 / 残差 / 初始化 cheat sheet 上线** —— 补 foundational 训练机理这个洞：**BatchNorm / LayerNorm / RMSNorm / GroupNorm** · **Pre-vs-Post-LN**（Xiong 梯度论证——Post-LN 为何需 warmup）· **DeepNorm / Sandwich / QK-Norm** · **残差连接 + 缩放**（LayerScale / ReZero / GPT-2 的 1/√(2N)）· **Xavier vs Kaiming**（ReLU 的因子 2，且守恒的量是二阶矩 E[y²] 而非 Var）· **μP**（宽度不变超参迁移）· **Fixup / NFNets / DyT**（归一化-free）· 以及 **covariate-shift 辨析**（Santurkar）。**双语**（中文 + EN），含可跑脚本（[`code/normalization.py`](docs/tutorials/code/normalization.py) —— from-scratch LN/RMSNorm vs `torch` + Pre/Post-LN 梯度 + Kaiming 二阶矩 + GPT-2 残差缩放，真机验证）和 25 高频题。跨模型审抓到 Pre/Post-LN 梯度 demo 的一个真方法学混淆（输出归一化假象），改用对 loss 稳健的顶/底梯度比。[`normalization_init_tutorial.html`](docs/tutorials/normalization_init_tutorial.html)。
 - **2026-06-18** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) ⚡ **第 26 篇 线性 / 稀疏注意力 cheat sheet 上线** —— 补上次二次 / 高效注意力这个全集到处引用却没系统讲的洞：**线性注意力**（kernel φ + 结合律 → 矩阵状态 RNN）· **SSM / Mamba**（selective S6）· **Mamba-2 / SSD**（1-半可分对偶 ≡ 结构化掩码线性注意力）· **DeltaNet / Gated DeltaNet**（改写式更新）· **chunkwise 并行**训练 · **可训练稀疏**（NSA 三分支 / MoBA / Lightning / DSA）· **混合架构**（Jamba / Hymba / Qwen3-Next / Kimi-Linear / MiniMax-01）。**双语**（中文 + EN），含可跑脚本（[`code/linear_sparse_attention.py`](docs/tutorials/code/linear_sparse_attention.py) —— `chunkwise ≡ recurrent` 等价 + delta-rule + block-sparse，真机验证）和 25 高频题，经多批 Codex GPT-5.5 xhigh 引用 / 数学 / 代码 / 答案 / 整体 + render 保真审定稿。[`linear_sparse_attention_tutorial.html`](docs/tutorials/linear_sparse_attention_tutorial.html)（[#22](https://github.com/wanshuiyin/ARIS-in-AI-Offer/pull/22) · [b84e913](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/b84e913)）。
 - **2026-06-13 → 06-14** — ![NEW](https://img.shields.io/badge/NEW-red?style=flat-square) 🧩 **两篇 must-know cheat sheet 上线：第 24 篇 LoRA / PEFT + 第 25 篇 RAG + 文本嵌入/检索** —— 全集到处在用、却从没讲过数学的两个 glaring hole。**LoRA/PEFT**：B=0 恒等起点 · α/r vs rsLoRA √r · 零延迟 merge · QLoRA（NF4 / 双重量化 / paged）· DoRA · 家族 vs Adapter / Prefix / Prompt / BitFit。**RAG**：双塔 · InfoNCE + 难负例 · Matryoshka · BM25 · HNSW · RRF 混合 · cross-encoder vs ColBERT late interaction · HyDE / Self-RAG / CRAG · GraphRAG · RAGAS。两篇均**双语**（中文 + EN），各含从零 PyTorch、可跑脚本（[`code/lora.py`](docs/tutorials/code/lora.py) · [`code/rag_embedding.py`](docs/tutorials/code/rag_embedding.py)，真机验证）和 25 高频题，经多轮 Codex GPT-5.5 xhigh 数学/代码 + render + EN 翻译保真审定稿。[`lora_peft_tutorial.html`](docs/tutorials/lora_peft_tutorial.html) · [`rag_embedding_retrieval_tutorial.html`](docs/tutorials/rag_embedding_retrieval_tutorial.html)。
 - **2026-06-08** — ![POLISH](https://img.shields.io/badge/POLISH-blue?style=flat-square) 🔧 [`tools/render_html.py`](tools/render_html.py) 在 frontmatter 检测前先剥掉开头的 UTF-8 BOM（[6cc4876](https://github.com/wanshuiyin/ARIS-in-AI-Offer/commit/6cc4876)）。
@@ -109,6 +110,7 @@
 | Topic | HTML 中文 | HTML EN | MD |
 |---|---|---|---|
 | **Attention 面试 Cheat Sheet** | [📄 中](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/attention_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/attention_tutorial_en.html) | [MD](docs/tutorials/attention_tutorial.md) |
+| **归一化 / 残差 / 初始化 (BatchNorm / LayerNorm / RMSNorm / Pre-vs-Post-LN / DeepNorm / QK-Norm / Xavier·Kaiming / μP)** | [📄 中](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/normalization_init_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/normalization_init_tutorial_en.html) | [MD](docs/tutorials/normalization_init_tutorial.md) |
 | **KL Divergence in RLHF (k1/k2/k3 · placement gradient bias)** | [📄 中](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/kl_divergence_rlhf_tutorial.html) | [📄 EN](https://wanshuiyin.github.io/ARIS-in-AI-Offer/tutorials/kl_divergence_rlhf_tutorial_en.html) | [MD](docs/tutorials/kl_divergence_rlhf_tutorial.md) |
 
 ### 🎯 Post-Training & Reasoning

docs/tutorials/code/README.md

@@ -16,6 +16,7 @@
 | `lora.py` | `LoRALinear`（B=0 起点 · α/r 与 rsLoRA 缩放 · merge/unmerge）+ `DoRALinear`（幅度-方向分解）+ 6 个 assert | [lora_peft_tutorial.md](../lora_peft_tutorial.md) | <5s |
 | `rag_embedding.py` | 双塔 `DualEncoder` + InfoNCE（in-batch / 难负例）+ BM25 + RRF 混合召回 + 6 个 assert | [rag_embedding_retrieval_tutorial.md](../rag_embedding_retrieval_tutorial.md) | <5s |
 | `linear_sparse_attention.py` | linear attention 递推/chunkwise 等价 + delta rule（DeltaNet 改写式更新）+ block-sparse top-k（NSA/MoBA select）+ 6 个 assert | [linear_sparse_attention_tutorial.md](../linear_sparse_attention_tutorial.md) | <5s |
+| `normalization.py` | LayerNorm/RMSNorm from-scratch vs `torch` + RMSNorm 去 re-centering + BatchNorm train≠eval + Pre/Post-LN 梯度 top-heavy + Kaiming/Xavier 二阶矩 E[y²] + GPT-2 残差 1/√(2N) + 6 个 assert | [normalization_init_tutorial.md](../normalization_init_tutorial.md) | <5s |
 
 ## 运行
 
@@ -29,6 +30,7 @@ python toy_mmdit_t2i_pipeline.py # 依赖 mmdit_block.py 在同目录
 python lora.py
 python rag_embedding.py
 python linear_sparse_attention.py
+python normalization.py
 ```
 
 每个脚本都自带 sanity check：shape 验证 + 数值合理性检查 + 必要时跟 PyTorch