tools: Omega-QVLA dit_svdquant_v1 pack → E0M3/UE4M3 converter + format doc + synthetic fixture - #179
Open
Lmy271828 wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Offline tooling to consume Omega-QVLA quantized packs (
dit_svdquant_v1, W4A4 GPTQ + DuQuant rotations for pi0.5) on FlashRT's SM110 E0M3 path:tools/convert_omega_pack_e0m3.py— re-quantizes each record's dequantized fp16 weight into the E0M3 operand layout (packed 4-bit[N, K/2]+ tile-interleaved UE4M3 SFB scales) viaquantize_e0m3_dynamic_sfa_fp16. Two--foldstrategies for the activation-side per-channel calibration table:none(S0) andmean(S1). Aux tensors (DuQuant rotations, permutation, scale tables) are copied through for the runtime consumer.tools/check_omega_e0m3_layer.py— single-layer cosine gate: converted E0M3 GEMM vs. fp16 reference and vs. Omega's own fake-quant (pure-torchemulatemode for pre-checks,kernelmode on hardware).tools/gen_omega_pack_fixture.py— synthetic miniature pack (schema-identical records, random-orthogonal rotation blocks, outlier-channel weights) so the full path is reviewable without the 4.8 GB real pack.docs/omega_pack_e0m3.md— format doc: record schema, the S0/S1 analysis, measured results, roadmap. Purely additive (tools/,docs/); no changes to kernels or frontends. Runtime consumption (flag-gatedweight_formatwiring) is a follow-up PR.Key finding: drop the calibration table (S0)
The pack ships a per-step, per-channel activation scale table. Folding its mean into the weights (S1) collapses on real hardware (cos 0.16 on q_proj): the fold presses per-16 block scales down to the UE4M3 subnormal floor (2⁻⁹). Dropping the table entirely (S0) wins — DuQuant's rotation+permutation already whitens channel magnitudes, and per-16 dynamic amax is the stronger quantizer. Measured on Thor (4 layers): S0 vs fp16 = 0.9924–0.9932, ≥ Omega's own fake-quant (0.9921–0.9928). Consequence: no per-step scale dispatch is needed on the runtime path, which simplifies the follow-up wiring PR.
Measured (Thor, pi0.5 LIBERO-10)
Reproduce (self-contained, no Omega-QVLA checkout needed)
Fixture validation on Thor: 4/4 records converted, consumer-level cosine vs fp16 = 0.9887/0.9890 (q_proj/down_proj), E0M3 0.19 ms/layer.
Non-goals
• Runtime weight_format wiring (next PR, per the roadmap in §5).
• SVDQuant low-rank epilogue fusion (rank = 0 in this pack; deferred).
• Per-step weight tables / activation scale dispatch (refuted by S0, §4).