Is your feature request related to a problem? Please describe.
I am unable to load ultra-large MoE quantized models like bearzi/Kimi-K2.7-Code-JANGTQ_K on Apple Silicon using oMLX. When attempting to load JANGTQ-formatted checkpoints, the engine fails during initialization with parameter mismatch errors (Received 540 parameters not in model: ... tq_packed, tq_norms, tq_bits). Standard MLX weight loaders lack the dequantization kernels required to handle JANGTQ packed tensors and its jang_config.json specifications.
Describe the solution you'd like
Support native loading and inference for JANGTQ (JANG Quantization) formatted models within oMLX. Specifically:
Parse jang_config.json configurations and recognize JANGTQ quantization attributes (tq_packed, tq_norms, tq_bits).
Integrate dequantization/fused kernels for JANGTQ packed MoE expert layers to enable efficient local execution of models like Kimi-K2.7-Code on Apple Silicon Mac devices.
Describe alternatives you've considered
Running via jangq runtime / Python scripts: Requires running a separate external server wrapper outside of oMLX's unified environment and loses oMLX's superior memory management/KV cache optimization.
Converting/Loading via standard GGUF or HF formats: Ultra-large models (~1T parameter MoE) are too memory-heavy in standard GGUF formats; JANGTQ offers a tailored ternary/low-bit packing scheme designed specifically for Apple Silicon unified memory constraints.
Additional context
Target Model: bearzi/Kimi-K2.7-Code-JANGTQ_K
Key Quantization Identifiers: Model config contains jang_config.json and tensor shapes packed with .tq_packed, .tq_norms, .tq_bits.
Reference Implementation: jjang-ai/jangq Repository
Environment: macOS (Apple Silicon), oMLX backend.
Is your feature request related to a problem? Please describe.
I am unable to load ultra-large MoE quantized models like bearzi/Kimi-K2.7-Code-JANGTQ_K on Apple Silicon using oMLX. When attempting to load JANGTQ-formatted checkpoints, the engine fails during initialization with parameter mismatch errors (Received 540 parameters not in model: ... tq_packed, tq_norms, tq_bits). Standard MLX weight loaders lack the dequantization kernels required to handle JANGTQ packed tensors and its jang_config.json specifications.
Describe the solution you'd like
Support native loading and inference for JANGTQ (JANG Quantization) formatted models within oMLX. Specifically:
Parse jang_config.json configurations and recognize JANGTQ quantization attributes (tq_packed, tq_norms, tq_bits).
Integrate dequantization/fused kernels for JANGTQ packed MoE expert layers to enable efficient local execution of models like Kimi-K2.7-Code on Apple Silicon Mac devices.
Describe alternatives you've considered
Running via jangq runtime / Python scripts: Requires running a separate external server wrapper outside of oMLX's unified environment and loses oMLX's superior memory management/KV cache optimization.
Converting/Loading via standard GGUF or HF formats: Ultra-large models (~1T parameter MoE) are too memory-heavy in standard GGUF formats; JANGTQ offers a tailored ternary/low-bit packing scheme designed specifically for Apple Silicon unified memory constraints.
Additional context
Target Model: bearzi/Kimi-K2.7-Code-JANGTQ_K
Key Quantization Identifiers: Model config contains jang_config.json and tensor shapes packed with .tq_packed, .tq_norms, .tq_bits.
Reference Implementation: jjang-ai/jangq Repository
Environment: macOS (Apple Silicon), oMLX backend.