[BlivionIaG]

Refresh 2026-11-11 (merged origin/rdna_extras 1136bb952b into the branch, no force)

The doc was written against 30d6e82dcd (2026-10-21). The branch was 2,166 commits behind, the merge was clean, and the doc was not current. The citation checker reported "21 citations, 0 missing" on the old text, but it only proves that a line exists, not that it says what the sentence claims. In the old text, nearly every fa_rdna2.cu cite had drifted by 13 to 250 lines. I re-read each cited line (new commit d98dfffc22). The body below is updated to match.

What was stale and is now rewritten in the README (and TESTPLAN where noted):

• The int4_fp16_block premise. The README said RDNA_ATTN "rejects every quantised KV dtype", listed supported_kv_cache_dtypes as auto/float16/bfloat16, and called PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 an unmerged proposal whose codec "is not on this branch". PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 (explore doc) and PR feat(rdna2): INT4 KV cache with FP16 block scales, natively read on the QSA path #138 (implementation, 8dec4520e4) are merged: rdna_attn.py:764-769 lists int4_fp16_block, and its own branch in __init__, forward and the writer runs before the FP16-only gate (rdna_attn.py:329-330), which every other quantised dtype still hits. KVQuantMode.INT4_FP16_BLOCK is at kv_cache_interface.py:61; the codec is at docs/explore/kv-int4-fp16-scales/reference_codec.py. The V620 numbers for INT4 are in the commit message of 8dec4520e4 (3.24x tokens, 16k/1k c=1 decode +11.2 %, c=8 aggregate decode -12.4 %, PPL 4.98 vs 4.94), so "FP8 is the only dtype measured on V620" is also gone.
• The geometry decision (item 4). int4_fp16_block chose option (a) and shows what an int8 port needs: a customize_spec branch that reserves the scale bytes (rdna_attn.py:861-876; the generic branch at :877-888 sizes content with no scale bytes) and backend-owned cache views (int4_fp16_kv_layout.py:44, :72). The sizing statement for int8 is marked [INFERENCE]; nothing has allocated an int8 cache on this backend.
• Registration (item 1). a948ee01de (2026-11-04) registered fa_rdna2_prefill_paged_varlen_int8 for a workspace test (torch_bindings.cpp:464-476). fa_rdna2_decode_paged_int8 still has no schema or ops.h declaration; reshape_and_cache_int8_rdna2 is registered and unused. The shim fa_rdna2_backend.py has only the two INT4 wrappers.
• The writer defect (item 3) is unchanged, now at fa_rdna2.cu:4905 / :4907 (was :4681 / :4683). The int32 slot_mapping check moved to :4933, the duplicate ops.h declarations to :505 / :815.
• The Triton writer now floors the scale at 1e-6 and rounds half away from zero before the int8 store; clamp [-128, 127]. The doc said "clamped to [127, -128]".
• "PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 describes this as native INT8 kernels." That was PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134's first draft; it was corrected on 2026-10-23 (997e37aa1a) and the merged README says what this doc says.
• New overlap. The merged INT4 README now has "The INT8 backport: what it actually costs" (items 1, 3, 5, 6, 7 of this doc at shorter length) and "Why INT8 is not the answer at this budget" (FP8 wins at 8 bits; a new INT8 format is not worth building on SNR alone). This doc still adds the cache-geometry mismatch, the read-path analysis (N1 to N3) and the wiring map, so it is not a duplicate; the README says so.
• The INT4 reader uses the same FP16-tile dequant (int4_load_half into sK/sV, fa_rdna2.cu:513-519) as the shipped INT8 branch, so an N1 rewrite would apply to both dtypes.
• TESTPLAN: writer cite fixed; the codec location fixed; the commands could not have run (bench throughput has no --concurrency, --kv-cache-dtype was passed twice, .venv/bin/python does not exist). They now use bench serve --max-concurrency 1 and one --kv-cache-dtype.
• Every cite is written as a full path:line, so check_citations.py now covers 62 citations (was 21; the bare :4457-style cites were invisible to it). It reports 62 checked, 0 missing, and I read each one against its sentence.

Re-run on 2026-11-11: the README's Reproducing the numbers snippet against the in-tree codec gives all eight SNR rows and bit counts exactly as published (33.415 / 44.022 at 8.250 bits for the shipped format; 33.415 / 44.027 at 8.125; 36.425 / 44.764 at 8.250; 39.475 / 45.594 at 8.500). pre-commit passes on the touched files. No GPU was used and no hardware measurement exists for INT8; the V620 cells in TESTPLAN stay empty.

Still a draft: this is a design document whose own text says a human has to defend every claim before it leaves draft. The branch also carries an empty wip: feat(rdna2): FA-RDNA2 int8/fp8 KV cache commit (44616b40d2) that I did not touch (no history rewrite).

The summary below is the original text (against 30d6e82dcd). Its line references and its points 1, 2 and 4 are superseded by this section and by the README.

---

Status: draft. This is a design document. Nothing in the serving path changes, no dtype is added, and every V620 cell in the test plan is empty on purpose. The long form is docs/explore/kv-int8-native/README.md; this body is the summary.

Why

INT8 KV cache is not a new capability in vLLM. int8_per_token_head ships (vllm/config/cache.py:54), the Triton backend serves it (vllm/v1/attention/backends/triton_attn.py:321), and the writer and its scale-cache views are tested (tests/quantization/test_per_token_kv_cache.py).

What does not exist is a way to serve that dtype from the RDNA2 / FA-RDNA2 backend. The INT8 FA-RDNA2 kernels are already in this tree — and they are unwired, unreachable, and their writer is defective.

This PR replaces a six-line backlog stub with the mapping, the measured answer to "what is the cheapest way to read an INT8 cache on gfx1030", and the format that should go with it.

What is already in the tree

Piece	Where (against origin/rdna_extras @ 30d6e82dcd)
the dtype, the mode, the Triton backend that serves it	vllm/config/cache.py:54, vllm/v1/kv_cache_interface.py:48, vllm/v1/attention/backends/triton_attn.py:321
the Triton writer (scale = absmax/127 per token-head)	vllm/v1/attention/ops/triton_reshape_and_cache_flash.py:272
the cache geometry: [K(hs) | K_scale(4 B) | V(hs) | V_scale(4 B)] per (head, slot)	vllm/v1/attention/backends/triton_attn.py:405-427
the FA-RDNA2 int8 decode / prefill / writer	csrc/rocm/fa_rdna2.cu:4457, :4270, :4699
the int8 dot primitive (v_dot4c_i32_i8)	csrc/rocm/w4a8_sdot4_rdna2.cuh:125-128 (PR #93)

What is missing or broken

1. The two read ops are not registered. Only the writer is in csrc/rocm/torch_bindings.cpp:543-546, and the comment above it says why.
2. The backend gate rejects every quantised KV dtype. vllm/v1/attention/backends/rdna_attn.py:283 and :601-605.
3. The writer writes the scale bytes at the wrong address. csrc/rocm/fa_rdna2.cu:4681 / :4683 put byte d of slot s into plane HEAD_DIM, slot s+d, instead of plane HEAD_DIM+d, slot s. 17 of the 19 distinct scale addresses in a page are written by more than one thread, and planes HEAD_DIM+1..+3 are never written at all.
4. The writer's cache geometry is not vLLM's geometry. It expects [2, num_blocks, H_kv, D+4, block_size]; vLLM allocates one packed tensor with the scale inline. split_kv_cache (vllm/v1/attention/ops/paged_attn.py:25-29) only resolves when the content dim is exactly head_size. This is the largest unknown in the port and it needs an explicit decision, not a patch.
5. Three different scale index conventions. The kernels index by logical sequence position (:443), the legacy bridge by physical (block, slot), vLLM by physical (block, slot, head). The fix is local: the decode kernel already has the physical block and slot at the load point.
6. The legacy Python reader is not the inverse of the writer — .view(torch.float32) reinterprets the slot dimension as bytes.
7. The legacy evidence is a throughput number (2.93 vs 2.91 tok/s at 5.89 GiB) and cannot distinguish a correct scale from a corrupted one. No writer-to-reader test has ever existed for this path.
8. Smaller: the int8 ops return a Tensor while every other fa_rdna2 op writes into out; the writer wants an int32 slot_mapping; csrc/rocm/ops.h declares reshape_and_cache_int8_rdna2 twice (:458, :763).

The native INT8 read path

The ISA study in the parent project already measured this (.omo/plans/rdna2-int8-isa-benchmark/isa-report.md, V620), so this PR does not guess:

Variant	What it does	D=256, seq=16384	D=128, seq=1024
fp16_ref	the FP16 ceiling	86.3 out/s (1.000×)	7679 (1.000×)
v1	int8→fp16 then v_dot2	75.7 (0.877×)	7184 (0.935×)
v2	v_dot4c_i32_i8 (sdot4)	134.5 (1.560×)	8548 (1.113×)
v3	packed dword load + VGPR dequant + v_dot2	190.4 (2.207×)	7618 (0.992×)

Three conclusions:

• The study's implementation choice is v3, not the native int8 dot. sdot4 was implemented and measured, and rejected for the QK dot at production occupancy ("we lose SDOT4's single-thread advantage when we parallelize across threads, which production does with 128 thr/CTA"). It is recorded here so it is not re-litigated.
• The pattern this tree ships today is the measured loser. The IS_INT8 decode branch (csrc/rocm/fa_rdna2.cu:443) is a per-element scalar dequantise into an FP16 shared tile — the v1/v0 family. The study: "The current fa_rdna2_decode_paged_int8 is leaving 30%+ on the table at the typical Qwen3.x head dim (128)."
• The one native-dot opportunity nobody has measured is the PV product: quantise P to int8 and use v_dot4c_i32_i8 for PV. The study says so twice — "The PV (output) update is scalar in all variants… not measured here."

Note that none of the FA-RDNA2 int8 kernels is a native INT8 dot today: the cache is INT8, the arithmetic is FP16 (fdot2 against an FP16 Q at :472-477). PR #134's description of this work as "native INT8 prefill and decode kernels" is not what the code does, and this PR says so explicitly.

The format: INT8 + FP16 block scale

The INT4 sibling of this idea is PR #134. Extended to INT8 it becomes:

Field	Value
Element	INT8, symmetric
Block scale	FP16, one per k values along the head dimension
Bits/value	8 + 16/k → 8.250 at k=64, 8.500 at k=32

Measured with PR #134's reference_codec.py (synthetic rows, not V620 measurements):

Format	bits/value	K-side dB	V-side dB
int8_per_token_head (shipped: one FP32 scale per row)	8.250	33.415	44.022
int8_fp16_block k=128 (same granularity, FP16 scale)	8.125	33.415	44.027
int8_fp16_block k=64	8.250	36.425	44.764
int8_fp16_block k=32	8.500	39.475	45.594

• FP16 instead of FP32 is free: 0.000 dB on K, 0.005 dB on V, 0.125 bits/value cheaper.
• At k=64 the block scale pays for itself: the same 8.250 bits/value as the shipped format, +3.01 dB on the K side.
• The INT4 packed-zero-point trick does not port to INT8: an 8-bit zero-point does not fit in an FP16 mantissa next to a usable scale, so the format is symmetric. Whether the asymmetry is worth its own 0.25 bits/value is an open, measurable question.
• A block scale is nearly free for the native path: the score becomes one scaled dot per block, ~0.06 extra ops per element.

Order of work

1. Fix the writer offsets and write the writer-to-reader test (never existed).
2. Decide the cache geometry and make writer, reader and cache spec agree; the scale index convention falls out of this.
3. Register the two ops, wire the binding shim, open the backend gate.
4. A/B int8_per_token_head against FP16 on the V620 — NIAH, PPL, KV capacity, 16k/1k and 32k cells, with output-token parity against the FP16 baseline, not just throughput.
5. Port the v3_128 pattern into the decode kernel and re-measure.
6. Only then: the native INT8 PV, and the int8_fp16_block format.

Why this is not a duplicate

• The shipped Triton INT8 path is untouched; this is about the RDNA2 backend serving the same dtype.
• PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 is a different element width at a different budget (4.500 bits/value). This PR's format is its INT8 sibling and deliberately reuses its codec, measurements and reasoning about grouping axes rather than forking them — and says so where the two disagree (the packed zero-point).
• PR feat(rdna2): opt-in W8A8 int8 sdot4 dense and MoE kernels for gfx1030 #93 uses the same instruction (v_dot4c_i32_i8) on a different tensor (activations for a weight GEMM).
• The ISA study in the parent project is the starting point, not something re-run here.
• KVarN (Muller et al., Huawei CSL,
  arXiv:2606.03458, reference
  implementation at huawei-csl/KVarN)
  is the strongest published KV-cache method and is not a duplicate of this
  PR, because it is a different method at a different budget: a Hadamard
  rotation and an iterative dual-axis variance normalisation in front of an
  asymmetric round-to-nearest step, with two scales and a zero-point per
  group of 128 elements, at 2.375 bits/value for the 2-bit configuration and
  3.375 for the released 4-bit-key preset. This PR is about the native
  INT8 read path on RDNA2 - whether an integer dot can reach the KV cache
  at all, given an FP16 query - plus an INT8 + FP16 block scale at 8.250
  bits/value. The two share the observation that a KV quantiser's cost is
  set by its read path, not by its bit width: measured on the same synthetic
  tensor, KVarN's 2-bit configuration reads back at 12.1 instructions per
  element against INT4's 8.1, because it applies two scales and a zero-point
  to every element, while its rotation is free (it rotates the query in and
  un-rotates the output). That is the same argument this PR makes about the
  INT8 path, arrived at from the published method's own code.

Tests run

No serving-path code changes, so no model evaluation applies. What was run:

python3 docs/explore/kv-int8-native/check_citations.py
# 21 citations checked against origin/rdna_extras, 0 missing

check_citations.py resolves every file:line reference in the README against the base commit and prints the cited source, so a stale reference is visible; it exits non-zero if one does not resolve. The fork's pre-commit hooks (ruff, markdownlint, SPDX, forbidden imports) pass.

The reference-codec numbers come from PR #134's committed codec, reproduced by the snippet in the README's Reproducing the numbers section.

AI assistance

This draft was written with AI assistance. Every file:line reference was checked against the base tree by a script that is committed here, and the numbers come from the committed codec rather than from the author. No hardware measurement was taken for this PR — the V620 cells in TESTPLAN.md are empty on purpose, and a human has to review and defend every claim above before this leaves draft.

[BlivionIaG]

Mapped this against rdna_extras while working on the INT4+FP16 KV exploration (#134), which assumes this PR is its host. Summary: the HIP side is already in this tree, so this is a Python-and-wiring job — but there is a writer defect in rdna_extras today that has to be fixed before any of it is correct, and the existing end-to-end evidence cannot tell a correct scale from a corrupted one.

Good news: the kernels are already here

origin/rdna_extras carries the whole INT8 FA-RDNA2 stack:

• csrc/rocm/fa_rdna2.cu — fa_prefill_paged_varlen_int8 (:4411), fa_rdna2_decode_paged_int8 (:4598), reshape_and_cache_int8_rdna2 (:4840), and the int8 split-K prefill kernels (:2476 for D=128, :2724 for D=256).
• The int8 kernels take the scale as a separate fp32 tensor k_scale_per_tok / v_scale_per_tok shaped [N_kv_total, H_kv], and look it up inline at the KV load point — k_scale_ptr[n_global * H_kv + h_kv] (:2572, :2813). No smem scale table, no fp16 workspace.

So the legacy humanwork-legacy commit 3baecdb516 was not the source of the kernels; the port only has to bring the bindings and the dispatch across.

What is missing to make it run

1. Op registration. fa_rdna2_prefill_paged_varlen_int8 and fa_rdna2_decode_paged_int8 are not in csrc/rocm/torch_bindings.cpp. Only the writer is (:543-546), with a comment that already says why:

   "Not wired up yet: RDNA_ATTN still rejects quantized KV caches, and fa_rdna2_decode_paged_int8 is not registered."

   The legacy tree reached these kernels through a load_inline JIT build; this tree loads them through torch.ops._rocm_C, so csrc/rocm/torch_bindings.cpp and vllm/v1/attention/ops/fa_rdna2_backend.py both have to change.

2. The backend gate. RdnaAttentionBackend._can_run_fa_rdna2 rejects every quantized KV dtype:

   # vllm/v1/attention/backends/rdna_attn.py:281
   if get_kv_quant_mode(self.kv_cache_dtype) != KVQuantMode.NONE:
       return False

   and supported_kv_cache_dtypes is ["auto", "float16"] (:583). Both need to change for int8_per_token_head.

3. The cache-layout contract. The int8 cache is interleaved — data and the four scale bytes of a slot share one page — so the page size in vllm/v1/kv_cache_interface.py has to account for the scale bytes, and the writer, the reader and the page spec all have to agree on one layout. See the two items below; this is the part that decides the design.

4. Op signature. The int8 ops return a Tensor while every other fa_rdna2 op writes into an out parameter. Worth normalising while the binding is being written, not after.

The defect to fix first

csrc/rocm/fa_rdna2.cu:4628 writes the four bytes of a slot's fp32 scale at:

k_dst[HEAD_DIM * block_size + slot_in_block + d] = ...

Under the declared layout — [num_blocks, H_kv, HEAD_DIM+4, block_size], row_bytes = (HEAD_DIM+4)*block_size, stride_d = block_size — plane p occupies offsets [p*block_size, p*block_size+block_size). So the offset above puts the four bytes of one slot's scale into four different slots of plane HEAD_DIM, rather than into four different planes of one slot. Simulating it for block_size=16:

slot s, byte j	writer writes	a reader needs
s=0, j=0	plane 128, slot 0	plane 128, slot 0 ✅
s=0, j=1	plane 128, slot 1	plane 129, slot 0 ❌
s=0, j=2	plane 128, slot 2	plane 130, slot 0 ❌
s=0, j=3	plane 128, slot 3	plane 131, slot 0 ❌

Consequences: 17 of the 19 distinct addresses in a page's scale region are written by more than one thread, and planes HEAD_DIM+2 and HEAD_DIM+3 are never written at all — plane HEAD_DIM+1 only receives spill from the last three slots of the page. The correct offset is

k_dst[(HEAD_DIM + d) * block_size + slot_in_block] = ...

The same expression appears for the value cache at :4630.

The reader is not the inverse of the writer

The legacy Python bridge in humanwork-legacy (vllm/v1/attention/backends/rdna_attn.py:383) does:

k_scale_per_slot = kv_cache[0, :, :, _hs:_hs + 4, :].view(torch.float32).squeeze(2)

.view(torch.float32) reinterprets the slot dimension as bytes, not the byte dimension. Even with the writer fixed, that does not recover the scales — it packs four consecutive slots' byte 0 into one fp32.

Three different scale index conventions

Worth settling in one place before writing code, because the three parties disagree:

Party	Scale index
The int8 kernels	logical sequence position — k_scale_ptr[n_global * H_kv + h_kv]
The legacy Python bridge	physical (block, slot)
The canonical Triton scale cache in this tree	physical (block, slot, head) — torch.as_strided(..., size=(num_blocks, block_size, nkv), ...) in vllm/v1/attention/backends/triton_attn.py:451

Physical and logical indexing differ whenever a block is partially filled, which is the normal case with chunked prefill. The port has to pick one and make the writer, the reader and the page spec agree on it.

The evidence gap

humanwork-legacy has tests/kernels/test_int8_prefill_correctness.py and test_int8_decode_correctness.py, and the end-to-end bench log is at .omo/plans/rdna2-int8-isa-benchmark/e2e-int8-bench.log on the parent project (2.93 output tok/s against a 2.91 tok/s FP16 baseline, at 5.89 GiB of KV cache).

That is a throughput measurement. It cannot distinguish a correct scale from a corrupted one, and with the writer bug above the scales are corrupted — a dequantised K of ~0 still produces plausible tokens per second. No test in the legacy tree exercises the writer-to-reader path, so the numerical correctness of the legacy INT8 path was never established.

The port therefore needs a correctness test that has never existed, not just a re-run of the throughput bench. Concretely: write K/V through reshape_and_cache_int8_rdna2, read the scale back out of the cache, and assert it equals the per-(token, head) amax/127 the writer computed.

Smaller items

• csrc/rocm/ops.h declares reshape_and_cache_int8_rdna2 twice (:458 and :763), the second one not behind a guard.
• The int8 work is a seven-commit chain, not a single commit, so the port should be reviewed as a series rather than as one drop-in.

Suggested order of work

1. Fix the writer offsets (:4628, :4630) and add the writer-to-reader correctness test.
2. Pick one scale index convention; make the writer, the reader and kv_cache_interface.py agree on it.
3. Register the two ops and wire the binding shim; open the backend gate and supported_kv_cache_dtypes.
4. A/B int8_per_token_head against FP16 on the V620: NIAH / long-context probes, PPL, KV capacity, the 16k/1k and 32k cells — with output-token parity against the FP16 baseline, not just throughput.
5. Only then decide whether the INT4 format in [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 is worth swapping in. It is a different format (asymmetric INT4 with a 4-bit zero-point packed into the low mantissa bits of an FP16 block scale, 4.500 bits/value at k=32, against 8.250 for this one), and the arithmetic behind that choice is written up in [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134.

Worth keeping the two apart: this PR is the host for a quantized KV cache in the RDNA2 backend, and the format question is a separate one. Landing INT8 first — and measuring it properly — is the cheaper step, and it gives the INT4 work a real baseline to beat.

[BlivionIaG]

The draft body is now a full proposal rather than a six-line backlog stub, and the branch carries docs/explore/kv-int8-native/{README.md,TESTPLAN.md,check_citations.py}.

Corrections and additions to my earlier mapping comment on this PR:

• Line numbers. The writer defect I quoted at fa_rdna2.cu:4628 / :4630 has moved with the base branch: on origin/rdna_extras @ 30d6e82dcd it is :4681 / :4683. Same defect, same fix — (HEAD_DIM + d) * block_size + slot_in_block.
• The biggest finding is new, and it is bigger than the writer bug. The FA-RDNA2 writer expects two tensors with a legacy geometry, [2, num_blocks, H_kv, D+4, block_size]. What vLLM allocates for int8_per_token_head is one packed tensor whose content dim carries the scale inline, [K(hs) | K_scale(4 B) | V(hs) | V_scale(4 B)] per (head, slot) (vllm/v1/attention/backends/triton_attn.py:405-427). The RDNA2 backend's own plumbing assumes the FP16 geometry — split_kv_cache (vllm/v1/attention/ops/paged_attn.py:25-29) only resolves when the content dim is exactly head_size — so the port needs an explicit decision about the cache layout, not just a patch to the offset.
• The kernels are not native INT8 dots. The IS_INT8 decode branch (fa_rdna2.cu:443) dequantises per element into an FP16 shared tile and dots with v_dot2_f32_f16 (:472-477); the int8 prefill kernel does the same (:2588-2621). The cache is INT8, the arithmetic is FP16. I have corrected the record here because PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 describes this work as "native INT8 prefill and decode kernels".
• The native-dot question has a measured answer, and it is not "use sdot4". The ISA study in the parent project (.omo/plans/rdna2-int8-isa-benchmark/isa-report.md) measured six dequant/dot patterns on the V620. Its verdict: the packed-load + VGPR-dequant + fdot2 pattern (v3) is the implementation choice at 2.207x the FP16 ceiling at D=256/seq=16384; sdot4 (v2) wins single-threaded at D=128 but loses at production occupancy; and the pattern this tree ships today is in the measured-losing v1/v0 family ("leaving 30%+ on the table at the typical Qwen3.x head dim (128)"). The one native-dot item nobody has measured is the PV product — quantise P to int8 and use v_dot4c_i32_i8 for PV. The document lays that out, including the two things it costs (P drops to int8 precision, and the V scale has to be folded into P normalised by the block maximum).
• The format. int8_fp16_block — INT8 with an FP16 scale per 64 values along the head axis — costs exactly what the shipped per-token-head scale costs (8.250 bits/value) and is +3.01 dB on the K side on PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134's reference codec. Measured today with that codec: shipped int8_per_token_head 33.415 dB (K) / 44.022 dB (V) at 8.250 bits/value, int8_fp16_block k=64 36.425 / 44.764 at 8.250, k=32 39.475 / 45.594 at 8.500. The INT4 packed-zero-point trick does not port to INT8 — an 8-bit zero-point does not fit in an FP16 mantissa next to a usable scale — so the format is symmetric, and whether the asymmetry earns its own 0.25 bits/value is left as an open, measurable question.

Two things worth flagging for whoever reviews this:

1. The V620 cells are empty. Nothing in this PR has been run on hardware. The SNR figures are arithmetic on synthetic rows from PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134's committed codec, and the ISA ratios are from a different harness (single-thread, compute-bound) that measured dequant/dot patterns, not this kernel.
2. A small correction to PR [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 while I was in its numbers. Its "INT8 family" table reports int8_e4m3_block k=32 as having 190 of 256 block scales go subnormal. That is the K-side count; on the V side the codec prints 247 of 256. The SNR numbers in that table are right. It is a one-line fix and I have not touched [explore] INT4 + FP16 block scales for the RDNA2 KV cache (NVFP4-inspired) #134 — its branch and PR are yours to approve.

The check_citations.py in the branch resolves every file:line reference in the README against a base commit and prints the cited source, so a stale reference is visible rather than plausible. It currently reports 21 citations, 0 missing.

[BlivionIaG]

Prior-work note added: KVarN (Huawei CSL).

The body now answers KVarN explicitly:
arXiv:2606.03458, with the reference
implementation at huawei-csl/KVarN.

It is not a duplicate of this PR — it is a different method at a different
budget (a Hadamard rotation and an iterative dual-axis variance normalisation
in front of an asymmetric RTN step, two scales and a zero-point per 128
elements, 2.375 bits/value for the 2-bit configuration and 3.375 for the
released 4-bit-key preset). This PR is about the native INT8 read path on
RDNA2 and an INT8 + FP16 block scale at 8.250 bits/value.

It is worth recording because it reaches the same conclusion this PR does,
from the published method's own code: a KV quantiser's cost is set by its
read path, not by its bit width. Measured on the same synthetic tensor,
KVarN's 2-bit configuration reads back at 12.1 instructions per element
against INT4's 8.1, because it applies two scales and a zero-point to every
element — while its rotation is free, because it rotates the query in and
un-rotates the output.

Nothing in the proposal changes.