Add fused DeepSeek4 optimization ops

[gianni-cor]

Summary

• Adds fused GGML ops for DeepSeek4 LID scoring, hyper-connection post mixing, and 4x4 Sinkhorn normalization.
• Wires the DeepSeek4 graph to use the fused ops instead of expanded reference subgraphs.
• Implements the fused ops for CPU, CUDA, Metal, and Vulkan, including Vulkan shader generation and Metal pipeline selection.
• Updates backend meta/RPC handling and adds bit-exact test-backend-ops coverage for the new ops.

Testing

• git diff --check tether/temp-9840...HEAD
• CPU: cmake -S . -B build-review -DLLAMA_BUILD_TESTS=ON -DLLAMA_BUILD_TOOLS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_SERVER=OFF -DLLAMA_BUILD_APP=OFF -DGGML_METAL=OFF -DGGML_VULKAN=OFF -DGGML_CUDA=OFF
• CPU: cmake --build build-review --target test-backend-ops -j 8
• CPU: ./build-review/bin/test-backend-ops -b CPU -o DSV4_LID_SCORE_BIT_EXACT,DSV4_HC_POST_BIT_EXACT,DSV4_HC_SINKHORN_BIT_EXACT (15/15 passed)
• Metal: cmake -S . -B build-review-metal -DLLAMA_BUILD_TESTS=ON -DLLAMA_BUILD_TOOLS=OFF -DLLAMA_BUILD_EXAMPLES=OFF -DLLAMA_BUILD_SERVER=OFF -DLLAMA_BUILD_APP=OFF -DGGML_METAL=ON -DGGML_VULKAN=OFF -DGGML_CUDA=OFF
• Metal: cmake --build build-review-metal --target test-backend-ops -j 8
• Metal: ./build-review-metal/bin/test-backend-ops -b MTL0 -o DSV4_LID_SCORE_BIT_EXACT,DSV4_HC_POST_BIT_EXACT,DSV4_HC_SINKHORN_BIT_EXACT (15/15 passed on Apple M4)

[aegioscy]

The matmul result indexer_kq is still fully materialized before ggml_dsv4_lid_score consumes it. Could we investigate a DeepSeek4-specific matmul epilogue that applies ReLU, multiplies by indexer_weights, and reduces over heads while accumulators are still resident? This should eliminate the largest LID intermediate, plus its layout/contiguous conversion and an extra kernel launch. The current op can remain as the fallback.

[aegioscy]

Could we benchmark this without volatile (or isolate the exact bit-exact requirement)? volatile on the accumulator and every inner-loop product can inhibit register allocation/vectorization and force unnecessary loads/stores on CPU. If strict ordering is required for backend bit-exact tests, a non-volatile fast path with an explicit deterministic reduction may preserve reproducibility at much lower cost.

[jpgaribotti]

Three new model-specific ops in the core ggml op enum. AGENTS.md explicitly says to "prefer reusing existing infrastructure over introducing new components" and to pause on changes that "add whole new subsystems." Adding DSV4_* (DeepSeek4-specific) entries to the global GGML_OP_* enum, RPC protocol, backend-meta, and all four backends is a large, invasive, model-specific addition. Upstream ggml generally resists model-specific fused ops in the core op set.

Could be worthwhile to consider if a more generic shape can be found for the ops. For example, DSV4_HC_SINKHORN is the most clearly generalizable.

Sinkhorn-Knopp normalization (alternating row/column normalization toward a doubly-stochastic matrix) is a standard, model-agnostic algorithm. It shows up in optimal-transport layers and in some MoE routing schemes. Nothing about the algorithm is DeepSeek4-specific - only the implementation is, in two ways:

• It's hardcoded to 4x4 (16 registers, float4 reductions, fully unrolled loops).
• The op takes eps and n_iter as fixed params, which is already generic.

A clean generalization would be ggml_sinkhorn(x, eps, n_iter) operating on the leading NxM of each [N, M, batch, 1] tensor, with the kernel special-casing small sizes (4x4, 8x8) for the register-resident fast path and falling back to shared memory for larger matrices. That would be a defensible standalone ggml op that other models could reuse.

DSV4_HC_POST is a batched small GEMM + rank-1 update, also somewhat generic and could at least be renamed.

[gagallo7]

Upstream already implemented some of this PR.

Looking at the next rebase target b10069, we have these new ops: https://github.com/ggml-org/llama.cpp/blob/178a6c44937154dc4c4eff0d166f4a044c4fceba/ggml/include/ggml.h#L573-L576

• GGML_OP_LIGHTNING_INDEXER: which, IIUC, would replace DSV4_LID_SCORE
• Same for DSV4_HC_SINKHORN -> GGML_OP_DSV4_HC_COMB
• Finally, GGML_OP_DSV4_HC_POST could replace DSV4_HC_POST

---

So I think this PR should instead target the temp-10069, and fill the gaps with the Metal kernels and Vulkan shaders. Otherwise, we will need to carry on competitive implementations.