Skip to content

tools: Omega-QVLA dit_svdquant_v1 pack → E0M3/UE4M3 converter + format doc + synthetic fixture - #179

Open
Lmy271828 wants to merge 1 commit into
flashrt-project:mainfrom
Lmy271828:pr1-omega-e0m3-converter
Open

tools: Omega-QVLA dit_svdquant_v1 pack → E0M3/UE4M3 converter + format doc + synthetic fixture#179
Lmy271828 wants to merge 1 commit into
flashrt-project:mainfrom
Lmy271828:pr1-omega-e0m3-converter

Conversation

@Lmy271828

Copy link
Copy Markdown

What

Offline tooling to consume Omega-QVLA quantized packs (dit_svdquant_v1, W4A4 GPTQ + DuQuant rotations for pi0.5) on FlashRT's SM110 E0M3 path:

  • tools/convert_omega_pack_e0m3.py — re-quantizes each record's dequantized fp16 weight into the E0M3 operand layout (packed 4-bit [N, K/2] + tile-interleaved UE4M3 SFB scales) via quantize_e0m3_dynamic_sfa_fp16. Two --fold strategies for the activation-side per-channel calibration table: none (S0) and mean (S1). Aux tensors (DuQuant rotations, permutation, scale tables) are copied through for the runtime consumer.
  • tools/check_omega_e0m3_layer.py — single-layer cosine gate: converted E0M3 GEMM vs. fp16 reference and vs. Omega's own fake-quant (pure-torch emulate mode for pre-checks, kernel mode on hardware).
  • tools/gen_omega_pack_fixture.py — synthetic miniature pack (schema-identical records, random-orthogonal rotation blocks, outlier-channel weights) so the full path is reviewable without the 4.8 GB real pack.
  • docs/omega_pack_e0m3.md — format doc: record schema, the S0/S1 analysis, measured results, roadmap. Purely additive (tools/, docs/); no changes to kernels or frontends. Runtime consumption (flag-gated weight_format wiring) is a follow-up PR.

Key finding: drop the calibration table (S0)

The pack ships a per-step, per-channel activation scale table. Folding its mean into the weights (S1) collapses on real hardware (cos 0.16 on q_proj): the fold presses per-16 block scales down to the UE4M3 subnormal floor (2⁻⁹). Dropping the table entirely (S0) wins — DuQuant's rotation+permutation already whitens channel magnitudes, and per-16 dynamic amax is the stronger quantizer. Measured on Thor (4 layers): S0 vs fp16 = 0.9924–0.9932, ≥ Omega's own fake-quant (0.9921–0.9928). Consequence: no per-step scale dispatch is needed on the runtime path, which simplifies the follow-up wiring PR.

Measured (Thor, pi0.5 LIBERO-10)

  • 252/252 records converted; single-layer gates pass (details: §4 of the format doc).
  • Downstream, with a runtime consumer wired to the converted artifact: LIBERO-10 ×500 paired success rate 90.4% vs. BF16 baseline 91.6% (McNemar p = 0.53, no significant difference), episode time 58 s vs. 148 s for the PyTorch fake-quant path (2.6×); the consumer's pybind kernels are CUDA-graph capturable (10-step denoise loop in a single graph, 50-episode check 45/50 ≈ eager). Evidence: https://github.com/Lmy271828/FlashRT/blob/libero-eval-fp4/docs/omega_pack_e0m3.md

Reproduce (self-contained, no Omega-QVLA checkout needed)

python tools/gen_omega_pack_fixture.py --out /tmp/fixture_pack.pt   # CPU, seconds                                                       
python tools/convert_omega_pack_e0m3.py --pack /tmp/fixture_pack.pt \                                                                    
   --out /tmp/fixture_e0m3.pt --fold none                           # Thor                                                               
python tools/check_omega_e0m3_layer.py --pack /tmp/fixture_pack.pt \                                                                     
   --mode kernel                                                    # Thor                                                               

Fixture validation on Thor: 4/4 records converted, consumer-level cosine vs fp16 = 0.9887/0.9890 (q_proj/down_proj), E0M3 0.19 ms/layer.

Non-goals
• Runtime weight_format wiring (next PR, per the roadmap in §5).
• SVDQuant low-rank epilogue fusion (rank = 0 in this pack; deferred).
• Per-step weight tables / activation scale dispatch (refuted by S0, §4).

@Lmy271828
Lmy271828 requested a review from LiangSu8899 as a code owner August 18, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant