fix(models): resolve verifier_norm class by verifier family - #1090
Open
minziyu wants to merge 1 commit into
Open
fix(models): resolve verifier_norm class by verifier family#1090minziyu wants to merge 1 commit into
minziyu wants to merge 1 commit into
Conversation
Some verifier families store the final RMSNorm weight in the Gemma convention — applied gain is (1 + w) — while verifier_norm was always constructed as a plain Qwen3RMSNorm (gain w). This includes the Gemma models (as covered for Gemma3 by vllm-project#892) and the Qwen3.5/Qwen3.8 family, whose Qwen3_5RMSNorm is an alias of vLLM's GemmaRMSNorm (vllm/model_executor/models/qwen3_5.py); transformers' Qwen3_5RMSNorm.forward likewise computes output * (1.0 + weight). The mismatch silently mis-scales the reconstructed verifier targets: on a Qwen3.8-27B verifier (V=248320), feeding real model.norm.weight and real lm_head rows through both variants gives rel-L2 0.51 vs the verifier's own final hidden state (KL ~0.48 nat on the reconstructed target distribution, top-1 prob 0.008 vs 0.071 true). Training converges regardless — argmax mostly survives the ~2x scale error — so nothing flags it. Likely (part of) what vllm-project#797 observed as "DFlash performs worse on Qwen3.5 than Qwen3". Generalizes vllm-project#892's Gemma3 norm-class selection into a shared resolve_verifier_norm_class helper: detection keys off the verifier's model_type / text_config.model_type (read from its config.json when resolvable) with an architectures fallback — not weight statistics, since in the Qwen3.8 checkpoint per-layer norm weights sit near zero (gemma-style storage) while the final norm weight sits near 1.0, so value-based heuristics would misfire on exactly the layer that matters. The checkpoint weight loads verbatim; the convention is applied in the norm class's forward, matching the verifier's own numerics. Verified end-to-end on the DSpark-on-Qwen3.8 pipeline (hidden-states extraction through serving) with the equivalent weight-fold variant of this fix; a unit test asserts Gemma3RMSNorm(w) == Qwen3RMSNorm(w+1). Refs vllm-project#892, vllm-project#797. Signed-off-by: minziyu <645657703@qq.com>
minziyu
requested review from
fynnsu,
orestis-z,
rahul-tuli and
shanjiaz
as code owners
September 7, 2026 09:35
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews
🔴 Require approval from approved reviewers listWaiting for any of
This rule is failing.All pull requests must have at least one approving review from a member of the approved reviewers list before merging.
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthroughChangesThe PR detects the verifier’s final RMSNorm convention from model metadata. DFlash now selects the matching norm class. Tests cover metadata detection, model construction, checkpoint loading, and numerical equivalence. Verifier Norm Convention
Merge Risk: ⚪ Minimal · up to DFlash now applies the verifier family’s RMSNorm gain convention while retaining checkpoint weights unchanged. The supported model-family paths and numerical behavior are covered, with no remaining current-head merge risk identified. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some verifier families store the final RMSNorm weight in the Gemma convention — applied gain is (1 + w) — while verifier_norm was always constructed as a plain Qwen3RMSNorm (gain w). This includes the Gemma models (as covered for Gemma3 by #892) and the Qwen3.5/Qwen3.8 family, whose Qwen3_5RMSNorm is an alias of vLLM's GemmaRMSNorm (vllm/model_executor/models/qwen3_5.py); transformers' Qwen3_5RMSNorm.forward likewise computes output * (1.0 + weight).
The mismatch silently mis-scales the reconstructed verifier targets: on a Qwen3.8-27B verifier (V=248320), feeding real model.norm.weight and real lm_head rows through both variants gives rel-L2 0.51 vs the verifier's own final hidden state (KL ~0.48 nat on the reconstructed target distribution, top-1 prob 0.008 vs 0.071 true). Training converges regardless — argmax mostly survives the ~2x scale error — so nothing flags it. Likely (part of) what #797 observed as "DFlash performs worse on Qwen3.5 than Qwen3".
Generalizes #892's Gemma3 norm-class selection into a shared resolve_verifier_norm_class helper: detection keys off the verifier's model_type / text_config.model_type (read from its config.json when resolvable) with an architectures fallback — not weight statistics, since in the Qwen3.8 checkpoint per-layer norm weights sit near zero (gemma-style storage) while the final norm weight sits near 1.0, so value-based heuristics would misfire on exactly the layer that matters. The checkpoint weight loads verbatim; the convention is applied in the norm class's forward, matching the verifier's own numerics.
Verified end-to-end on the DSpark-on-Qwen3.8 pipeline (hidden-states extraction through serving) with the equivalent weight-fold variant of this fix; a unit test asserts Gemma3RMSNorm(w) == Qwen3RMSNorm(w+1).
Refs #892, #797.