Skip to content

Commit 692b1cb

Browse files
committed
research: record frozen multilingual confirmation
1 parent a510dc9 commit 692b1cb

6 files changed

Lines changed: 12608 additions & 14 deletions

File tree

README.md

Lines changed: 31 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,20 +3,24 @@
33
RecurQuant is a reproducible research harness for **persistent recurrent-state
44
quantization in Gated DeltaNet language models**.
55

6-
> **Current status:** research infrastructure and diagnostic pilot. There is no
7-
> validated quantization method, memory reduction, or speedup result yet.
6+
> **Current status:** the frozen diagnostic candidate passed its untouched
7+
> confirmation trace. This is not yet a validated general method, realized
8+
> memory reduction, speedup, or novelty result.
89
910
The first calibration/development pilot found substantial layer heterogeneity.
1011
At a 4.22-bit average payload, retaining only Gated DeltaNet layer 0 at INT8 and
1112
using INT4 for the other 17 layers reduced worst-5% token KL by 79.8% on a
1213
retrieval-style trace and 62.2% on a code-style trace relative to uniform INT4.
13-
These are short synthetic traces, not a benchmark or generalization result.
14+
On the untouched multilingual trace, the same frozen plan reduced worst-5%
15+
token KL by 77.8% and increased top-1 agreement from 25.0% to 59.4%. These are
16+
short synthetic traces, not a benchmark or generalization result.
1417

1518
## Research question
1619

17-
Can sub-8-bit storage of Gated DeltaNet's fixed recurrent matrix state use its
18-
decay and write dynamics to preserve difficult long-context behavior better than
19-
uniform quantization at the same modeled bit budget?
20+
Can sub-8-bit storage of Gated DeltaNet's fixed recurrent matrix state allocate
21+
precision from query-weighted read sensitivity to preserve difficult
22+
long-context behavior better than uniform quantization at the same modeled bit
23+
budget?
2024

2125
[Qwen3.5-0.8B-Base](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base) is the first
2226
target. Its language model repeats three Gated DeltaNet layers followed by one
@@ -52,7 +56,9 @@ The narrower hypothesis under investigation is **precision allocation for the
5256
persistent Gated DeltaNet matrix state**, conditioned on Gated DeltaNet dynamics
5357
and compared at an equal bit budget. See
5458
[the claim boundary](research/CLAIM_BOUNDARY.md) and
55-
[pilot protocol](research/PILOT_PROTOCOL.md).
59+
[pilot protocol](research/PILOT_PROTOCOL.md). The documented experiment trail
60+
preserves the [failed signals and replacement](research/EXPERIMENT_001_SIGNAL_PIVOT.md)
61+
and the [untouched confirmation](research/CONFIRMATION_001.md).
5662

5763
The user-suggested
5864
[Gated DeltaNet-2 paper](https://arxiv.org/abs/2605.22791) reinforces why erase,
@@ -75,6 +81,24 @@ uv pip install --python .venv\Scripts\python.exe -e ".[dev]"
7581
The model experiment is intentionally separate from the unit-test suite because
7682
it downloads approximately 1.75 GB of public model weights.
7783

84+
## Reproduce the frozen confirmation
85+
86+
The script pins the model revision and records the environment, token digest,
87+
state layout, metrics, and canonical evidence hash:
88+
89+
```powershell
90+
.venv\Scripts\python.exe scripts\run_qwen35_smoke.py `
91+
--upgrade-layers 0 --low-bits 4 --high-bits 8 `
92+
--group-size 128 --rounding nearest `
93+
--prefill-tokens 32 --decode-tokens 32 `
94+
--prompt-profile multilingual `
95+
--output artifacts\multilingual-confirmation.json
96+
```
97+
98+
This reruns the already disclosed confirmation profile; it is a reproducibility
99+
check, not a new held-out test. The recorded result and its limitations are in
100+
[Confirmation 001](research/CONFIRMATION_001.md).
101+
78102
## Research discipline
79103

80104
- Model and tokenizer revisions are pinned in evidence artifacts.

0 commit comments

Comments
 (0)