Add blockwise FP8 training recipe for linears and MoE grouped experts

[iamzainhuda]

Summary

Supersedes #3228 (rebased onto latest main, and extended to cover MoE expert grouped GEMMs in addition to linears).

• Add Float8BlockwiseLinear wrapper backed by torchao.prototype.blockwise_fp8_training.linear, selectable via Float8LinearConverter.Config(recipe_name="blockwise"). Blockwise linears require bias-free layers with in/out features divisible by block_size (128) and SM90+.
• Add recipe_name to Float8GroupedExpertsConverter.Config ("fp8_rowwise" default, "fp8_blockwise" new). The blockwise recipe routes routed-expert grouped GEMMs through torchao's blockwise FP8 grouped MM (1x128 activation / 128x128 weight scaling, DeepSeek V3 style) via Float8TrainingRecipe.FP8_BLOCKWISE. Token-group padding to the 128 scaling block happens inside the torchao op, so no dispatcher-level padding swap is needed.
• Add deepseek_v3_16b_fp8_blockwise and deepseek_v3_debugmodel_fp8_blockwise configs that apply blockwise FP8 to both dense/MoE-adjacent linears and routed experts (lm_head and router.gate filtered).
• Add scripts/fp8_blockwise/compare_deepseek_v3_16b.sh for the 8-GPU BF16 vs blockwise FP8 loss-convergence comparison.

Requires a torchao build with torchao.prototype.moe_training blockwise FP8 support (Float8TrainingRecipe.FP8_BLOCKWISE), which is stacked on top of pytorch/ao#4551.

Test Plan

• pytest tests/unit_tests/test_quantization.py (new test_fp8_blockwise_applied_by_model_registry)

DeepSeek V3 16B loss convergence: 8x H100, 1000 steps, seed 42

STEPS=1000 NGPU=8 SEED=42 ./scripts/fp8_blockwise/compare_deepseek_v3_16b.sh -- default deepseek_v3_16b parallelism (EP=8, Interleaved1F1B PP), C4 dataset, blockwise FP8 applied to all 159 eligible linears plus routed-expert grouped GEMMs (DeepGEMM backend).

step	BF16 loss	blockwise FP8 loss
100	6.48479	6.46466
200	5.58810	5.60036
300	5.15687	5.14813
400	4.86677	4.86455
500	4.74816	4.74271
600	4.61650	4.61059
700	4.41522	4.41110
800	4.30919	4.30734
900	4.22516	4.22701
1000	4.16875	4.16650

Blockwise FP8 tracks BF16 within noise for the entire run (final delta 0.0002, FP8 slightly ahead at most checkpoints).

Throughput note: this run was ~7.4k tps for FP8 vs ~9.5k tps for BF16. The 16B config only compiles the loss, so quantization ops run eager; grouped-GEMM performance work is tracked separately in torchao (native blockwise grouped GEMM kernels).

DeepSeek V3 debugmodel: 1 GPU, 30 steps, seed 42

step	BF16 loss	blockwise FP8 loss
5	4.51185	4.50946
10	3.70587	3.70533
15	3.29148	3.28895
20	3.03678	3.03468
25	2.98922	2.98764
30	2.93086	2.93221

[pytorch-bot]

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.