dasLLAMA vulkan: record-once decode, 3B resident arming; dasSpirv f16acc coopmat

[borisbat]

Three slices from the Vulkan tier / tensor-extension arc, each measured and gated:

1. Record-once resident decode — one submit per token. The token cmd records once per set_layer/set_cls epoch; a decode step is two host memcpys (x, rope row — copied in-cmd from host-visible staging), 3 uint position words per layer, and one submit. Previously every token re-recorded all 333 dispatches, refilled every meta buffer, and paid 5 submits (4 of them synchronous staging round-trips for constant scalars). Also fixes a latent stage-mask gap (the cmd's transfer copies were ordered only by compute-to-compute barriers) and groups q|k|v and gate|up into shared barrier levels (bit-exact; measured perf-neutral, kept as dependency documentation). Measured on tinyllama Q8 resident: 115.7 to 129.9 tok/s (+12%), logits bit-identical (DECHASH equal), submits/tok 5 to 1. New lifetime vk_dispatch_count / vk_submit_count instruments (all dispatches already funnel through one helper). 34/34 test_vulkan_tier synthetic suite green.

2. 3B-class dense models can arm the resident driver. The first mid-size dense model hit two artificial walls: the arena shard cap was a 2 GiB literal (the device reports maxStorageBufferRange = 4 GiB — now queried), and the auto-cap ctx floor of 4096 rejected the 3B's computed 3429-ctx fit (DASLLAMA_GPU_MIN_CTX overrides; default unchanged). Llama-3.2-3B Q8 now arms: 28 layers, dim 3072, ctx 3429, pp512 ~1157 tok/s resident (for reference: llama.cpp Vulkan coopmat-on runs 984 on the same card/model).

3. dasSpirv: f16 cooperative-matrix accumulator + coopmatConvert. coopmatAcc_f16_16x16 (one coopmat_info row — the MulAdd/Store emit arms were already structural), MulAdd/Store overloads, and coopmatConvert = componentwise OpFConvert over the tile then OpStore (llama.cpp's f16acc cast-then-store form). Golden fixture + spirv-val arm added to tests/spirv/test_coopmat.das; the full 338-test spirv suite stays byte-identical. Exercised on a 3060 Ti in the mul_mm probe arc (compiles, runs, validates bit-clean).

Context for reviewers: the f16acc rail was built while porting llama.cpp's mul_mm coopmat design; the measured conclusion of that arc is that dp4a-sdot4 remains the strongest Q8_0 GEMM on Ampere-class Vulkan (our sdot4 3.8 TFLOP/s vs llama.cpp's coopmat MUL_MAT 1.79 in isolation via test-backend-ops), so no kernel swap ships here — the emitter capability lands for the K-quant/f16-model follow-ups.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XmDdP6jrhyRkz1UaKaP6v6

[Copilot]

Pull request overview

This PR advances the Vulkan resident-driver path in dasLLAMA and extends dasSpirv cooperative-matrix support to match llama.cpp’s f16-accumulator flow, with test coverage to keep SPIR-V output validated.

Changes:

• Refactors resident decode to record the full token command buffer once per set_layer / set_cls epoch, reducing per-token work to host memcpys + per-layer position word updates + a single submit; adds lifetime dispatch/submit counters.
• Removes two “artificial wall” caps for mid-size dense models by querying maxStorageBufferRange and allowing an env override for the resident-driver min context floor.
• Adds f16 accumulator coopmat support in dasSpirv (coopmatAcc_f16_16x16, emitter recognition, and coopmatConvert emission) plus a validating golden test.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/spirv/test_coopmat.das	Adds a new validation case asserting the expected op mix for the f16-accumulator + widening-to-f32 store path.
tests/spirv/_spirv_common.das	Introduces a new coopmat fixture kernel and *_words() provider for the f16-accumulator + convert sequence.
modules/dasSpirv/spirv/spirv_emit.das	Teaches the SPIR-V emitter about coopmatAcc_f16_16x16 and emits coopmatConvert via OpFConvert + OpStore.
modules/dasSpirv/spirv/spirv_builtins.das	Adds the f16 accumulator coopmat type and builtin stubs for MulAdd/Store plus coopmatConvert.
modules/dasLLAMA/dasllama/dasllama_math_vulkan.das	Implements record-once resident decode, fixes/clarifies barriers, adds dispatch/submit counters, and queries device storage-buffer range cap.
modules/dasLLAMA/dasllama/dasllama_common.das	Adds DASLLAMA_GPU_MIN_CTX override for the resident-driver min-ctx floor and uses it in the resident upload decision.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[borisbat]

Fixed in e190f99 - the warning now reports both quant and scale byte counts.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

modules/dasLLAMA/dasllama/dasllama_math_vulkan.das:4385

• These memcpy calls strip const from the source pointers via addr<void? -const>(...). Since memcpy accepts const sources elsewhere in the codebase, this can be tightened to avoid unnecessary const-laundering and make the intent (read-only source) explicit.

        memcpy(g_rd.x_host.mapped, addr<void? -const>(x[0]), int(g_rd.dim * 4l))
        memcpy(g_rd.cos_host.mapped, addr<void? -const>(cossin[0]), int(g_rd.head_size * 4l))

[borisbat]

Fixed in 41440a1 - non-numeric (to_int -> 0) now keeps the default floor; only positive values clamp to >=256.

[borisbat]

Fixed in 41440a1 - the arm now validates via coopmat_info that dst is an f32 accumulator and src an f16 accumulator of the same tile shape before emitting; golden suite re-run green.