i2_s SIGSEGVs at n_ubatch >= 32: BLAS backend dequantises by a row stride 4x the real packed row

[EUPHEMEME]

Summary

Any i2_s model SIGSEGVs during prompt processing as soon as n_ubatch >= 32, on any build that links a BLAS backend — which is the default on macOS via Accelerate. Single-token generation is unaffected, which is why this doesn't show up in tg benchmarks.

Repro

BitNet-b1.58-2B, i2_s:

llama-bench -m ggml-model-i2_s.gguf -ngl 0 -p 64 -ub 32   # SIGSEGV
llama-bench -m ggml-model-i2_s.gguf -ngl 0 -p 64 -ub 16   # fine

The threshold is exactly min_batch = 32 in ggml_backend_blas_device_supports_op — a backend-selection boundary, not a stack limit.

Cause

i2_s packs four 2-bit weights per byte but declares blck_size = 1, type_size = 1 (ggml.c, [GGML_TYPE_I2_S]). So

result->nb[1] = result->nb[0] * (result->ne[0] / ggml_blck_size(type));   // = ne0 bytes

while a real packed row is ne0/4. The row stride is intentionally 4x too large, and every BitNet-aware consumer compensates at the use site — src0_row + ir0 * nb01 / 4 appears verbatim three times in ggml_compute_forward_mul_mat. ggml_nbytes carries the matching correction (nbytes / 4 + 32).

That convention is self-consistent until a generic path touches i2_s. ggml_backend_blas_device_supports_op accepts any src0 whose type has a non-NULL to_float:

(src0->type == GGML_TYPE_F32 || ggml_get_type_traits(src0->type)->to_float != NULL);

i2_s qualifies, and the backend then dequantises row-wise by the raw nb01, walking four times too far and off the end of the tensor.

There is also no correct i2_s BLAS path even in principle: dequantize_row_i2_s takes a fourth argument (the scale) and is cast to the three-argument ggml_to_float_t, so the scale is never passed. Even if the stride fit, the dequantised weights would be unscaled — and BLAS has no way to apply the act_sums / +1-offset correction that the generic mul_mat wrapper applies for these types.

Fix

Exclude I2_S/TL1 from the BLAS backend and let the generic per-row path handle them, which it already does correctly. Two-line guard in the GGML_OP_MUL_MAT case. I'll open a PR.

Measured after the fix (M5 Max, -ngl 0), comparing against -ub 16, previously the only working setting:

model	-ub 16	-ub 512	gain
BitNet-b1.58-2B i2_s	8.70 t/s (pp512)	22.41 t/s	2.6x
Falcon3-10B-Instruct-1.58bit i2_s	1.69 t/s (pp256)	7.68 t/s	4.5x

Correctness, not just non-crash: greedy (--temp 0 --seed 1), the same prompt at -ub 16 / -ub 32 / -ub 512 produces byte-identical output, and the model correctly answers a fact-recall question that requires a 46-token prompt to have been prefilled correctly. -ub 16 is the pre-existing known-good path, so identity against it is the meaningful check. Verified with GGML_BLAS both ON and OFF.

Verification note

The bug is confirmed present at the currently pinned submodule commit 390c3077 by source inspection — min_batch = 32 and the to_float != NULL qualification are intact at ggml-blas.cpp:416-422, and blck_size = 1 plus the 4-argument to_float cast are intact at ggml.c:931-936.

The fix was built and measured against the earlier pinned tree 1f86f058 (build 3962), because current main does not build on Apple silicon for unrelated reasons. I have not been able to run current main end-to-end, so I'd appreciate a sanity check from someone who can.

[EUPHEMEME]

Here is the patch, so it's applyable without waiting on venue questions.

It's against ggml/src/ggml-blas/ggml-blas.cpp in the pinned submodule (Eddie-Wang1120/llama.cpp). Two-line guard plus the reasoning as a comment:

@@ -420,6 +420,22 @@ static bool ggml_backend_blas_device_supports_op(ggml_backend_dev_t dev, const s
             // TODO: find the optimal value
             const int64_t min_batch = 32;
 
+            // FIX (i2_s batched, 2026-08-05): i2_s/tl1 must never reach BLAS.
+            // They qualify below via `to_float != NULL`, but neither the stride
+            // nor the signature holds up:
+            //   - they declare blck_size = 1, type_size = 1, so nb01 is 4x a
+            //     real packed row; the row-wise dequant here walks off the end
+            //     of the tensor (SIGSEGV at exactly n_ubatch >= min_batch).
+            //   - dequantize_row_i2_s takes a fourth argument (the scale) and
+            //     is cast to the 3-argument ggml_to_float_t, so the scale is
+            //     never passed and the result would be garbage even if it fit.
+            // There is no correct BLAS path for these types; the generic
+            // ggml_compute_forward_mul_mat path handles them (and applies the
+            // act_sums / scale correction that BLAS has no way to apply).
+            if (src0->type == GGML_TYPE_I2_S || src0->type == GGML_TYPE_TL1) {
+                return false;
+            }
+
             return ggml_is_contiguous(src0) &&
                    ggml_is_contiguous(src1) &&
                    src1->type == GGML_TYPE_F32 &&

Where would you like the PR? The fix lands in the submodule repo rather than this one, and Eddie-Wang1120/llama.cpp's master is plain upstream llama.cpp — the i2_s work is on a feature branch. Rather than guess which one and open it against the wrong base, tell me the branch you want and I'll send it. Happy to send it to ggml-org/llama.cpp instead if you'd rather the guard live upstream of the fork, though the I2_S/TL1 types don't exist there.

The guard is deliberately narrow: it only short-circuits GGML_OP_MUL_MAT for these two types and leaves every other type's BLAS path untouched, so it can't regress f16/q4/q8 throughput.