Summary
PR #3022 added per-tensor scale (global_scale) support for NVFP4 quantization on CUDA and CPU backends. However, the Metal backend explicitly rejects it:
https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/quantized.cpp#L1725-L1730
if (mode_ == QuantizationMode::Nvfp4 &&
static_cast<int>(inputs.size()) > base_size) {
throw std::runtime_error(
"[QQMatmul] Global scale (tensor-scale nvfp4) is not supported "
"on the Metal backend.");
}
Impact
Without tensor-scale support, NVFP4 on Metal has ~137x less dynamic range than NVIDIA Blackwell (unsigned UE4M3 vs signed E4M3 scales), which was the original concern in #2962. The fix (PR #3022) landed for CUDA/CPU but Metal users still hit the error.
This blocks NVFP4 quantization for Apple Silicon users running MoE models (DeepSeek-V3/V4, GLM-5.1, etc.) where tensor-scale is critical for expert routing accuracy.
Request
Please add Metal kernel support for consuming the global_scale inputs in the qqmm / fp_quantized Metal kernel paths, matching the CUDA/CPU behavior from PR #3022.
Environment
- MLX commit: 973e27f
- Platform: Apple Silicon (M-series)
- Backend: Metal
Summary
PR #3022 added per-tensor scale (global_scale) support for NVFP4 quantization on CUDA and CPU backends. However, the Metal backend explicitly rejects it:
https://github.com/ml-explore/mlx/blob/main/mlx/backend/metal/quantized.cpp#L1725-L1730
Impact
Without tensor-scale support, NVFP4 on Metal has ~137x less dynamic range than NVIDIA Blackwell (unsigned UE4M3 vs signed E4M3 scales), which was the original concern in #2962. The fix (PR #3022) landed for CUDA/CPU but Metal users still hit the error.
This blocks NVFP4 quantization for Apple Silicon users running MoE models (DeepSeek-V3/V4, GLM-5.1, etc.) where tensor-scale is critical for expert routing accuracy.
Request
Please add Metal kernel support for consuming the global_scale inputs in the
qqmm/fp_quantizedMetal kernel paths, matching the CUDA/CPU behavior from PR #3022.Environment