M10.5: Bumblebee quantized inference integration

[ausimian]

Summary

Closes the gap M10 left open: Native.quantized_matmul is now reachable from Nx.Defn.jit-traced Axon forward passes, and a quantized Qwen3-0.6B greedy-decodes end-to-end through stock Bumblebee.Text.generation/4.

• Emily.Quantization.dequantize_defn/1 — defn-native MLX affine dequantize built from Nx.right_shift / bitwise_and / multiply / add. Supports bits ∈ {2, 4, 8}.
• Emily.Quantization.Layers.quantized_dense/4 — Axon-compatible layer op. Nx.dot(x, dequantize_defn(qw)) replaces MLX's fused quantized_matmul (two kernels vs. one; M11 closes the gap).
• Emily.Quantization.Transform.quantize/3 — graph rewriter + model-state quantizer mirroring Axon.Quantization.quantize/2. Rewrites every :dense node to :quantized_dense and swaps kernels for %QuantizedWeight{}.
• :qwen3_quant_full opt-in conformance — loads dense Qwen3-0.6B, quantizes via Transform.quantize/3 (bits=4, group_size=128, transpose=true), greedy-decodes 32 tokens, pins continuation text as regression gate.

Approach chosen: defn-native dequantize (Option 1 from PLAN.md M10.5). Rejected Option 2 (fork Nx.Defn.Evaluator — fragile) and Option 3 (upstream Nx custom-op hook — too slow).

Scope reductions from original PLAN.md M10.5

• AWQ safetensors loader deferred. On closer inspection AWQ groups along the in axis while MLX's transpose=false path expects groups along the stored last axis (out). Correct conversion requires transposing packed tensors, unpacking qzeros into biases, and mapping HF param names to Bumblebee names — tractable but additional scope. The from-dense path validates the full defn-integration pipeline end-to-end.
• Conformance oracle is Emily's own first-run output (same discipline as qwen3_full_test.exs) rather than MLX Python output on AWQ weights.

Full details in PLAN.md §M10.5.

Test plan

• mix precommit — 263 tests + 66 properties + 1 doctest, 0 failures, credo clean
• mix test --only qwen3_quant_full — Qwen3-0.6B quantized greedy decode matches pinned reference (~14s after model cache)
• 26 new unit/property tests across test/emily/quantization/:
  • dequantize_defn_test.exs — element-wise equality with QuantizedWeight.to_dense/1 across all (bits, group_size) combos
  • layers_test.exs — quantized_dense/4 transpose=true/false, defn composition
  • transform_test.exs — graph rewrite + state quantize + MLP round-trip under both Nx.Defn.Evaluator and Emily.Compiler