docs(tutorials): full-collection resweep — 236 fixes across 20 tutorials (GPT-5.6-sol find + Claude adversarial verify) - #31
Merged
Conversation
…als (GPT-5.6-sol find + Claude adversarial verify) A full re-audit of the 20 non-diffusion tutorials (training fundamentals, attention/long-context, RLHF/KL/OPD, inference systems, PEFT, reasoning/agents, multi-agent/self-evolving, RAG/VLM) — motivated by the reviewer model upgrading to GPT-5.6-sol since these tutorials last shipped under GPT-5.5. Two-stage pipeline, not a single pass: 1. GPT-5.6-sol (Codex, fresh thread per file, from training knowledge only) scanned all 20 files for factual/mathematical/citation/mechanism errors that survived the earlier GPT-5.5-era review → 240 candidate findings. 2. Independent Claude adversarial verification (different model family, fresh context per file, explicitly instructed to try to REFUTE each claim before accepting it) re-derived every finding from scratch against the actual tutorial text → 191 CONFIRMED, 45 NEEDS_NUANCE (softened/ qualified rather than flatly wrong), 4 REFUTED (original text correct, left untouched). 236 fixes applied; the 4 refuted findings prove the verification pass wasn't a rubber stamp. Highlights: - StarPO (RAGEN, arXiv 2504.20073) acronym: two independent verification rounds gave opposite answers (80% confidence each way) — resolved by fetching the paper abstract directly. It's State-Thinking-Actions-Reward Policy Optimization; the tutorial's internal inconsistency is fixed. - Real code bugs, not just prose: E4M3FN encoder round-trip bug (biased_e boundary condition wrongly saturated legal values 256-448 to 448); RowParallelLinear used the local shard's fan_in instead of global in_features in Kaiming init (variance inflated by tensor-parallel degree); train_step_4d crashed on non-final pipeline-parallel stages (empty-list torch.stack); GRPO KL loss used a length-biased global token-mean instead of per-completion normalization; DoRALinear was missing device/dtype propagation (same class of bug already fixed in the sibling LoRALinear, caught as a follow-up and re-verified on Server5). - Corrected mechanism-level misconceptions: DeepSeek-V3 routing (bias added after sigmoid affinity, not before; missing routed_scaling_factor≈2.5), MLA absorb trades HBM traffic for extra latent-space FLOPs (not free), ZeRO++ hpZ/qgZ mechanics, GPipe/1F1B activation-memory conventions, Mixtral parameter accounting (now matches official 46.7B/12.9B exactly), BLIP-2/Qwen-VL/LLaVA-NeXT connector architectures, Unigram forward-vs- Viterbi semantics carried over from the tokenization tutorial's own review, Xiong et al. Pre-LN/Post-LN gradient-imbalance interpretation, and citation/venue/year corrections (MoA ICLR 2025, AMSGrad arXiv id restored, FlashAttention-3 authorship, OCP MX spec year, etc.). - Where the first-pass reviewer itself was later shown wrong (Qwen vocab padding, this StarPO dispute), the fix reflects the corrected answer, not the first guess — corrections flow both ways between model families. Mechanics: every touched .md got its EN edition fixed in parallel (natural- language translation of the same substantive correction, not machine translation), both HTMLs re-rendered, both sidecars updated with a new audit-round entry (old rounds preserved) and a fresh source_sha256. Several fix agents proactively found and corrected duplicate restatements of the same error elsewhere in the same file, to avoid leaving a newly-introduced internal contradiction. Gate: tools/verify_reviews.py --mode strict --reproduce → 59 OK / 0 WARN / 0 FAIL / 3 EXEMPT / PASS. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
wanshuiyin
added a commit
that referenced
this pull request
Jul 13, 2026
…als + What's New entry for the prior 20-file resweep (#32) Second run of the full-collection resweep (see 59636aa / PR #31 for the first 20-file batch): the 8-file diffusion/generative-media cluster — 3d_generation, diffusion_distillation, diffusion_foundations, diffusion_post_training, flow_matching, image_generation_systems, vae_vqvae_vqgan, video_generation — was the one part of the collection still on GPT-5.5-era review. As of this PR, all 28 tutorials are under GPT-5.6-sol. Same two-stage pipeline: GPT-5.6-sol found 67 candidate errors (fresh thread per file) → an independent Claude pass adversarially re-verified each one (try to refute before accepting) → 55 CONFIRMED + 11 NEEDS_NUANCE fixed, 2 REFUTED correctly left untouched. Notably higher P0 density than the first batch — diffusion-model math has more subtle failure modes than the training/ systems/agent tutorials audited in round one. Highlights: - The single most-recurring error: conflating a "conditional interpolation path is a straight line" (true by construction, e.g. x_t=(1-t)x_0+tx_1) with "the marginal ODE trajectory the model learns is a straight line" (generally false under independent pairing — the marginal velocity field is a conditional expectation over all pairings sharing x_t, so it curves; this is exactly why Reflow exists). Recurred in diffusion_foundations (4 locations) and flow_matching (2 locations), each independently confirmed and fixed rather than resolved once and left inconsistent elsewhere. - Real code bugs: iCT's Pseudo-Huber constant was missing the paper's √D scaling (understating it ~50x at CIFAR-10 scale); its noise schedule used i.i.d.-sampled-then-sorted lognormal σ instead of the paper's deterministic Karras/EDM grid; FSQ's bound function was missing the parity-offset/atanh shift needed for even quantization levels; LFQ's entropy-regularization loss maximized per-sample entropy (pushing every bit toward 0.5 — the opposite of confident quantization) instead of minimizing per-sample entropy while maximizing batch-marginal entropy; DDPO's pseudocode divided by a deterministic (zero-variance) terminal transition, producing NaN. - Mechanism-level corrections: ADD's distillation loss was repeatedly described as teacher-output pixel MSE across 7+ locations in diffusion_distillation (it's a re-noise-then-single-teacher-denoise score-distillation signal — the file's own §4.1 already had this right, used as the alignment anchor); DMD2 was recommended as "GAN-free" when its core change from DMD v1 is adding a GAN loss; SD3's RF paper had its time direction backwards (t=0 is data, t=1 is noise, not the reverse); SD/SDXL's middle UNet block was said to have no cross-attention when it does (SpatialTransformer / UNetMidBlock2DCrossAttn); SDXL-Turbo's resolution was wrong (512px, not 1024px — that's SD3-Turbo); SVD's conditioning mechanism was misattributed (4+4=8 channel latent concat + CLIP cross-attn, not a mask-based 2C+1 scheme); Custom Diffusion was mischaracterized as a LoRA-DreamBooth variant when it directly optimizes cross-attention K/V (full-rank, no low-rank parametrization). - 3 cross-file consistency flags caught in a final orchestrator pass: two fix agents correctly declined to touch content outside their assigned tutorial even when they spotted the same class of error recurring elsewhere (SD3-Turbo mislabeled as a Reflow example in diffusion_foundations when it's LADD; OpenMagViT2 cited as an FSQ usage example in vae_vqvae_vqgan when it's LFQ per that file's own table) — swept up here before merge rather than left as latent inconsistencies. Mechanics: every touched .md got its EN edition fixed in parallel, both HTMLs re-rendered, both sidecars updated with an appended audit-round entry (old rounds preserved) and a fresh source_sha256. flow_matching's standalone runnable script was checked and needed no changes (independently re-verified via direct math re-derivation, not just trusted from the audit JSON). Also adds the README What's New entry (both editions) for the prior 20-file resweep (PR #31 / 59636aa), which had landed without one. Gate: tools/verify_reviews.py --mode strict --reproduce → 59 OK / 0 WARN / 0 FAIL / 3 EXEMPT / PASS. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-collection resweep: 240 candidates → 236 verified fixes across 20 tutorials
Since these tutorials last shipped, the reviewer model upgraded from GPT-5.5 to GPT-5.6-sol — this PR is a full re-audit under the stronger reviewer, covering everything except the diffusion/generative-media cluster (audited separately, unchanged here).
Two-stage pipeline (not a single pass)
The 4 refuted findings are the proof this wasn't a rubber stamp — the verifier pushed back where the first pass was wrong.
Highlights
RowParallelLinearKaiming init used the local shard'sfan_ininstead of globalin_features(variance inflated by the tensor-parallel degree)train_step_4dcrashed on non-final pipeline stages (torch.stack([]))DoRALinearwas missing device/dtype propagation — the same bug class already fixed in the siblingLoRALinear; caught as a follow-up and re-verified on a real GPU boxrouted_scaling_factor≈2.5), MLA absorb trades HBM traffic for extra latent-space FLOPs (not free), ZeRO++ hpZ/qgZ mechanics, GPipe/1F1B memory conventions, Mixtral parameter accounting (now matches the official 46.7B/12.9B exactly), BLIP-2/Qwen-VL/LLaVA-NeXT connector architectures, Xiong et al. Pre-LN/Post-LN gradient-imbalance interpretation, plus citation/venue/year fixes (MoA → ICLR 2025, AMSGrad arXiv id restored, FlashAttention-3 authorship, OCP MX spec year, etc.)Mechanics
.mdgot its EN edition fixed in parallel (natural-language translation of the same substantive correction, not MT)source_sha256python3 tools/verify_reviews.py --mode strict --reproduce→ 59 OK · 0 WARN · 0 FAIL · 3 EXEMPT · PASS🤖 Generated with Claude Code