[crispyging]

What this adds

A native GGUF load path for the Qwen4Exp (Flash-Next) architecture — load_format=qwen4exp_gguf — so a mixed-quant GGUF shard set runs through the fork's existing kernels instead of requiring a dequantized checkpoint or an alternative runtime.

Formats handled

part of the artifact	representation
dense projections, norms, routers, embeddings	decoded to fp16 (fp32 for A_log) at load, bounded row chunks
routed MoE experts	stay packed in VRAM as Q4_K / Q5_K / Q5_1 / Q8_0 blocks, TP-sharded on the output rows (w13) and on the packed input width (w2)
per-layer embedding table (PLE)	stays packed as IQ4_NL in rank-local pinned host memory, decoded on demand through UVA

Correctness-relevant details

• GGUF dimensions are innermost-first; reversing them yields (output, input) directly. No transpose — a transpose passes any square-matrix assertion and silently corrupts every non-square one.
• Zero-centered (Gemma-style) norms subtract 1: A_log = log(-ssm_a) with >= 0/non-finite rejected.
• GDN value heads are grouped then tiled by the inverse permutation, applied consistently to QKV V rows, conv1d value channels, ssm_alpha/ssm_beta matrix rows and the ssm_dt.bias vector; ssm_conv1d gains the unsqueeze(1) to (channels, 1, kernel).
• The loader fails closed: unmapped or missing tensors, gate/up quant-type mismatch, malformed or incomplete shard sets, and conflicting PLE environment settings all raise instead of loading a partial model.

Tests

125 passed in the loader/decoder suite, all CPU, no GPU or network required:

• per-format packed decoders compared against the official gguf package oracle on frozen fixtures (IQ4_NL codebook indices, sign masks, scales, subnormals, tile tails, row alignment), plus float64 accumulation bounds for the matmul references;
• GGUF v3 shard manifest validation against a real completed four-shard set, including the general.architecture-only-in-shard-1 pattern that llama.cpp emits;
• the full 1224 -> 923 tensor mapping verified against a meta-device Qwen4ExpForCausalLM (native parameter existence, constructor gates, final namespace), with one whole-inventory count assertion;
• TP1 and TP2 load_model through the real model registry in a separate CPU process, asserting exception messages rather than classes.

A toy 4-layer artifact (GDN, GDN+PLE, Q5_K/Q8_0 experts, QSA) exercises the loader end to end; its base config ships as a fixture so the suite runs from a clean checkout.

Validation

Beyond the CPU suite the path was run as a served engine on 4x gfx1030 with tensor parallelism 4, dtype=float16: weights load, all four workers bind the pinned IQ4_NL PLE, the RDNA2 GDN prefill kernel and FlashAttention are selected, and generated output passes a semantic/tool-call gate plus a benchmark-side output-diversity check.

Known limitation, unchanged by this PR: gfx1030 has no integer dot-product instruction, so the packed expert MMQ path emulates dp4a with four multiply-adds. Prefill throughput is therefore below what an int4-to-fp16 GEMM lane reaches on the same cards; a half2 dot path for the packed experts is the natural follow-up.