docs(tutorials): post-ship final review — DoRA axis bug fix + RAG terms + 2026 frontier enrichments (#24/#25) - #17
Merged
Conversation
… bug + RAG terms + frontier enrichments A fresh hostile-reviewer pass (Codex GPT-5.5 xhigh, re-deriving from scratch) caught a REAL bug in the shipped LoRA sheet that survived 4 review rounds: - DoRA magnitude axis was per-INPUT (norm dim=0) but real DoRA / HF PEFT use per-OUTPUT (torch.linalg.norm(weight, dim=1), weight-normalization style). Fixed §6 formula+code+comment, Q17, and code/lora.py (DoRALinear dim=0->dim=1). Re-ran on Server5 (identity |Δ| now 1.19e-7); Codex re-verified CORRECT. - §2.2: hedged "A≠0 ⇒ ∂L/∂B≠0" (degenerate g=0 / x∈null(A)). RAG sheet — two terminology corrections: - HyDE retrieves a hypothetical DOCUMENT/passage (not "answer"); §7 + Q18. - CRAG scores RETRIEVED-doc relevance/confidence (not "recall", untestable at inference without a labeled set); §7 + Q24. Compact 2026-frontier enrichments (both editions): - LoRA: S-LoRA/Punica multi-LoRA serving (Q23); LoRA Learns Less and Forgets Less (Biderman 2024, arXiv 2405.09673) (Q25); QDoRA (§6). - RAG: SPLADE learned-sparse (§5); ColBERTv2/PLAID + single-vs-multi-vector tradeoff (§6); Contextual Retrieval / late chunking (§10); BEIR/MTEB (§8.3). All 10 new citations cross-model-verified (ColBERTv2 title corrected). Both editions (中文 + EN) updated in lockstep; gate: 51 OK / 0 FAIL / 3 EXEMPT. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A user-requested final adversarial pass over the two newly-shipped cheat sheets (#24 LoRA/PEFT, #25 RAG) — fresh hostile-reviewer Codex GPT-5.5 xhigh, re-deriving everything from scratch. It caught a real bug that survived 4 prior review rounds, plus 2 minor RAG terminology errors, and adds compact 2026-frontier coverage.
🔴 The real bug (LoRA DoRA)
The DoRA magnitude–direction decomposition used per-input magnitude (
weight.norm(dim=0)), but real DoRA / HF PEFT (DoraLinearLayer.get_weight_norm→torch.linalg.norm(weight, dim=1)) and weight normalization use per-output (one magnitude per output neuron). The earlier reviews verified the decomposition was self-consistent (identity-start still holds) but never checked it against the reference convention.code/lora.pyDoRALinear(dim=0→dim=1).|Δ|now1.19e-7, all 6 asserts pass.🟡 Minor fixes
🟢 Compact 2026-frontier enrichments (both editions)
All 10 new citations cross-model-verified (ColBERTv2 title corrected to "Efficient and Effective"). Both 中文 + EN editions updated in lockstep (mechanical mirror verified).
Verification
verify_reviews.py --mode strict --reproduce: all 4 changed tutorials 1 OK each; full repo 51 OK · 0 FAIL · 3 EXEMPT → PASS🤖 Generated with Claude Code