feat: make Qwen3.5 quickstart installable

Author: Labeeb2339

CHANGELOG.md

@@ -12,8 +12,11 @@ software, so compatibility can still change between development versions.
 - `PackedRecurrentStateCache` for keeping recurrent states packed between layer
   calls, with opt-in evidence recording and exact resident-byte accounting.
 - A guarded Qwen3.5 cache factory, pinned Qwen3.5 quickstart, and compatibility
-  checks for the tested Transformers minor release and eager, single-device
+  checks for the tested Transformers release and eager, single-device
   inference path.
+- A reusable frozen v0.2 mixed-policy cache helper and an installed
+  `recurquant qwen35` workflow shared with the source-tree quickstart. Uniform
+  INT4 remains available only as an explicitly named stress baseline.
 - Frozen MBPP calibration and development evaluation workflows with pinned
   dataset/model revisions, prepared token manifests, canonical evidence hashes,
   equal-byte baselines, and resumable per-task checkpoints.
@@ -25,9 +28,10 @@ software, so compatibility can still change between development versions.
 - Scale-storage emulation now uses the declared physical FP16 or FP32 format;
   superseding diagnostic results retain the earlier record instead of rewriting
   it.
-- The Transformers dependency is constrained to `>=5.14.1,<5.15` while the
-  package depends on that minor release's internal linear-attention cache
-  contract.
+- The Transformers dependency is pinned to exactly `5.14.1` while the alpha
+  package depends on that release's internal linear-attention cache contract.
+- FP16 scale storage is identified as the evaluated default. FP32 scale storage
+  remains supported for experiments but is not covered by full-model evidence.
 
 ### Known limitations
 

CONTRIBUTING.md

@@ -53,7 +53,7 @@ In particular:
   them.
 
 The current supported package boundary is recorded in
-[docs/compatibility.md](docs/compatibility.md). A new model, Transformers minor
+[docs/compatibility.md](docs/compatibility.md). A new model, Transformers
 release, execution backend, generation mode, or hardware target is unsupported
 until it has a regression test and clearly scoped full-model evidence.
 

README.md

@@ -20,33 +20,48 @@ The commands are short; the first run still needs to download the pinned model
 and tokenizer. Python 3.11 and a CUDA GPU are recommended for the evaluated
 path.
 
+Windows PowerShell:
+
 ```powershell
 git clone https://github.com/Labeeb2339/recurquant.git
 cd recurquant
 py -3.11 -m venv .venv
 .\.venv\Scripts\python.exe -m pip install .
-.\.venv\Scripts\python.exe examples\qwen35_quickstart.py --max-new-tokens 16
+.\.venv\Scripts\recurquant.exe qwen35 --max-new-tokens 16
+```
+
+macOS or Linux:
+
+```bash
+git clone https://github.com/Labeeb2339/recurquant.git
+cd recurquant
+python3.11 -m venv .venv
+.venv/bin/python -m pip install .
+.venv/bin/recurquant qwen35 --max-new-tokens 16
 ```
 
-On macOS or Linux, replace `.\.venv\Scripts\python.exe` with
-`.venv/bin/python`. The runnable source is
-[`examples/qwen35_quickstart.py`](examples/qwen35_quickstart.py); read the
+The installed command and
+[`examples/qwen35_quickstart.py`](examples/qwen35_quickstart.py) call the same
+implementation. The default is the frozen v0.2 mixed policy: layer 0 at INT8
+and the remaining recurrent layers at INT4. Uniform INT4 is retained only as an
+explicit stress baseline via `--policy uniform-int4-stress`. Read the
 [compatibility contract](docs/compatibility.md) before using a different model,
 Transformers version, device layout, or generation mode.
 
 ## Use it in Python
 
-This example keeps Gated DeltaNet layer 0 at INT8 and the other 17 recurrent
-layers at INT4, matching the frozen development policy. Remove `layer_specs`
-for uniform INT4.
+This example uses the reusable frozen v0.2 helper, which keeps Gated DeltaNet
+layer 0 at INT8 and the other 17 recurrent layers at INT4. The generic
+`create_qwen35_packed_cache()` factory remains available for controlled policy
+experiments.
 
 ```python
 import warnings
 
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-from recurquant import QuantizationSpec, create_qwen35_packed_cache
+from recurquant import create_qwen35_v02_mixed_cache
 
 MODEL_ID = "Qwen/Qwen3.5-0.8B-Base"
 REVISION = "dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68"
@@ -74,12 +89,7 @@ model = AutoModelForCausalLM.from_pretrained(
 ).to(device)
 model.eval()
 
-cache = create_qwen35_packed_cache(
-    model,
-    bits=4,
-    group_size=128,
-    layer_specs={0: QuantizationSpec(bits=8, group_size=128)},
-)
+cache = create_qwen35_v02_mixed_cache(model)
 inputs = tokenizer("Explain recurrent-state quantization simply.", return_tensors="pt")
 inputs = inputs.to(device)
 continuation = []
@@ -102,10 +112,11 @@ print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
 print(cache.storage_summary())
 ```
 
-`create_qwen35_packed_cache()` rejects unsupported Transformers versions,
-non-eager attention, training mode, multi-device placement, and incompatible
-Qwen configurations early. The returned cache exposes exact live tensor byte
-accounting through `storage_summary()`.
+`create_qwen35_v02_mixed_cache()` and `create_qwen35_packed_cache()` reject
+unsupported Transformers versions, non-eager attention, training mode,
+multi-device placement, and incompatible Qwen configurations early. The
+returned cache exposes exact live tensor byte accounting through
+`storage_summary()`.
 
 ## What is physically smaller
 
@@ -150,10 +161,12 @@ Important boundaries:
 
 The supported public surface is deliberately narrow:
 
-- Python `>=3.11` and `transformers>=5.14.1,<5.15`;
+- Python `>=3.11` and exactly `transformers==5.14.1` for this alpha;
 - text-only Qwen3.5 hybrid models with `linear_attention` and `full_attention`
   layer types;
-- physical INT4 or INT8 recurrent-state payloads with FP16 scales;
+- physical INT4 or INT8 recurrent-state payloads; FP16 scales are the evaluated
+  default, while FP32 scales are supported as an experimental, unevaluated
+  option;
 - eager, evaluation-only, single-device inference; and
 - explicit `past_key_values=cache` model calls.
 

docs/compatibility.md

@@ -10,15 +10,27 @@ model quality, speed, or production readiness.
 | Area | Current evidence | Support boundary |
 |---|---|---|
 | Python | The full MBPP development run used Python `3.11.15`. Packaging CI is configured to smoke-test Python `3.11` and `3.13` on Linux, plus Python `3.11` on Windows. | Python `>=3.11` is declared. Full-model numerical evidence is limited to `3.11.15`. |
-| Transformers | `5.14.1` | The dependency is deliberately constrained to `transformers>=5.14.1,<5.15`. Other minor versions are unsupported until their internal cache contract is tested. |
+| Transformers | `5.14.1` | The alpha dependency is deliberately pinned to `transformers==5.14.1`. Every other release is unsupported until its internal cache contract is tested. |
 | Model | `Qwen/Qwen3.5-0.8B-Base` at revision `dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68` | No other checkpoint, revision, model family, or recurrent architecture has full-model evidence. |
 | Model execution | BF16 weights, batch size one, evaluation mode, `trust_remote_code=False`, eager attention | Use `attn_implementation="eager"`. Flash, SDPA, and other attention implementations are not validated. |
 | Full-run environment | PyTorch `2.11.0+cu128`, CUDA runtime `12.8`, NVIDIA driver `592.15`, NVIDIA GeForce RTX 5070 Laptop GPU, recorded platform `Windows-10-10.0.26200-SP0` | CPU and other accelerator support is limited to unit or API smoke tests; the public numerical result was not replicated there. |
-| Packed formats | Physical INT4 and INT8 recurrent-state payloads with FP16 scales | The packed cache does not accept other payload widths. |
+| Packed formats | Physical INT4 and INT8 recurrent-state payloads. FP16 scales are the evaluated default. | FP32 scales are supported for experiments but have no full-model fidelity evidence. The packed cache does not accept other payload widths. |
 
 The exact model and software provenance is recorded in
 [`evidence/mbpp-v02-development.json`](../evidence/mbpp-v02-development.json).
 
+## Installed quickstart
+
+`recurquant qwen35` and `examples/qwen35_quickstart.py` use one shared
+implementation. Both default to the frozen v0.2 mixed policy: model layer 0 at
+INT8 and every other recurrent layer at INT4, with group size 128 and FP16
+scales. `--policy uniform-int4-stress` is retained only for reproducing the
+uniform INT4 stress baseline.
+
+The command downloads the pinned model and tokenizer unless
+`--local-files-only` is supplied. It performs manual greedy decoding; the
+full-checkpoint quality of free-running generations has not been evaluated.
+
 ## Generation and cache modes
 
 The following paths have direct evidence:

examples/qwen35_quickstart.py

@@ -1,100 +1,6 @@
-"""Run a pinned Qwen3.5 model with RecurQuant's packed recurrent-state cache."""
-
-from __future__ import annotations
-
-import argparse
-import warnings
-
-import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-from recurquant import create_qwen35_packed_cache
-
-MODEL_ID = "Qwen/Qwen3.5-0.8B-Base"
-MODEL_REVISION = "dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68"
-
-
-def _arguments() -> argparse.Namespace:
-    parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument(
-        "--prompt",
-        default="Explain recurrent-state quantization in two sentences.",
-    )
-    parser.add_argument("--max-new-tokens", type=int, default=32)
-    parser.add_argument("--device", choices=("auto", "cpu", "cuda"), default="auto")
-    return parser.parse_args()
-
-
-def _device(requested: str) -> torch.device:
-    if requested == "auto":
-        return torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    if requested == "cuda" and not torch.cuda.is_available():
-        raise RuntimeError("--device cuda was requested, but CUDA is not available")
-    return torch.device(requested)
-
-
-def _model_dtype(device: torch.device) -> torch.dtype:
-    if device.type != "cuda":
-        return torch.float32
-    if torch.cuda.is_bf16_supported():
-        return torch.bfloat16
-    warnings.warn(
-        "CUDA BF16 is unavailable; falling back to FP16. RecurQuant's public "
-        "full-model fidelity evidence has not been validated for FP16 weights.",
-        RuntimeWarning,
-        stacklevel=2,
-    )
-    return torch.float16
-
-
-def main() -> None:
-    args = _arguments()
-    if args.max_new_tokens <= 0:
-        raise ValueError("--max-new-tokens must be positive")
-
-    device = _device(args.device)
-    dtype = _model_dtype(device)
-    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, revision=MODEL_REVISION)
-    model = AutoModelForCausalLM.from_pretrained(
-        MODEL_ID,
-        revision=MODEL_REVISION,
-        dtype=dtype,
-        attn_implementation="eager",
-    ).to(device)
-    model.eval()
-
-    cache = create_qwen35_packed_cache(model, bits=4, group_size=128)
-    encoded = tokenizer(args.prompt, return_tensors="pt").to(device)
-    generated: list[torch.Tensor] = []
-
-    with torch.inference_mode():
-        output = model(**encoded, past_key_values=cache, use_cache=True)
-        for step in range(args.max_new_tokens):
-            next_token = output.logits[:, -1, :].argmax(dim=-1, keepdim=True)
-            generated.append(next_token)
-            reached_eos = (
-                tokenizer.eos_token_id is not None
-                and bool((next_token == tokenizer.eos_token_id).all().item())
-            )
-            if reached_eos or step + 1 == args.max_new_tokens:
-                break
-            output = model(input_ids=next_token, past_key_values=cache, use_cache=True)
-
-    generated_ids = torch.cat(generated, dim=1)
-    print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
-
-    summary = cache.storage_summary()
-    print(f"resident_recurrent_state_bytes={summary['resident_bytes']}")
-    print(
-        "full_precision_equivalent_recurrent_state_bytes="
-        f"{summary['full_precision_equivalent_bytes']}"
-    )
-    print(
-        "largest_materialized_recurrent_state_bytes="
-        f"{summary['largest_materialized_state_bytes']}"
-    )
-    print(f"resident_compression_ratio={summary['resident_compression_ratio']:.3f}x")
+"""Run the installed RecurQuant Qwen3.5 quickstart workflow."""
 
+from recurquant.qwen35_quickstart import main
 
 if __name__ == "__main__":
-    main()
+    raise SystemExit(main())

pyproject.toml

@@ -36,8 +36,8 @@ dependencies = [
   "safetensors>=0.5",
   "torch>=2.7",
   # PackedRecurrentStateCache subclasses an internal cache layer. Keep the
-  # tested minor pinned until its compatibility contract is covered in CI.
-  "transformers>=5.14.1,<5.15",
+  # alpha on the exact tested release until another release is covered in CI.
+  "transformers==5.14.1",
 ]
 
 [project.optional-dependencies]

src/recurquant/__init__.py

@@ -8,14 +8,15 @@
     quantize_dequantize,
     quantize_pack,
 )
-from .qwen35 import create_qwen35_packed_cache
+from .qwen35 import create_qwen35_packed_cache, create_qwen35_v02_mixed_cache
 
 __all__ = [
     "PackedQuantizedTensor",
     "PackedRecurrentStateCache",
     "QuantizationResult",
     "QuantizationSpec",
     "create_qwen35_packed_cache",
+    "create_qwen35_v02_mixed_cache",
     "quantize_dequantize",
     "quantize_pack",
 ]

src/recurquant/cli.py

@@ -1,4 +1,4 @@
-"""Command-line entry points that do not require a model download."""
+"""Command-line entry points for RecurQuant."""
 
 from __future__ import annotations
 
@@ -8,6 +8,7 @@
 import torch
 
 from .quantization import QuantizationSpec, quantize_dequantize
+from .qwen35_quickstart import add_qwen35_arguments, run_qwen35_quickstart
 
 
 def _demo(args: argparse.Namespace) -> int:
@@ -47,6 +48,13 @@ def build_parser() -> argparse.ArgumentParser:
     demo.add_argument("--key-dim", type=int, default=16)
     demo.add_argument("--value-dim", type=int, default=16)
     demo.set_defaults(handler=_demo)
+
+    qwen35 = subparsers.add_parser(
+        "qwen35",
+        help="Run the pinned Qwen3.5 model with the packed recurrent-state cache.",
+    )
+    add_qwen35_arguments(qwen35)
+    qwen35.set_defaults(handler=run_qwen35_quickstart)
     return parser
 
 

src/recurquant/qwen35.py

@@ -20,8 +20,8 @@
 else:
     Qwen35Source = object
 
-_SUPPORTED_TRANSFORMERS_REQUIREMENT = "transformers>=5.14.1,<5.15"
-_SUPPORTED_TRANSFORMERS_SPEC = SpecifierSet(">=5.14.1,<5.15")
+_SUPPORTED_TRANSFORMERS_REQUIREMENT = "transformers==5.14.1"
+_SUPPORTED_TRANSFORMERS_SPEC = SpecifierSet("==5.14.1")
 _SUPPORTED_LAYER_TYPES = frozenset({"linear_attention", "full_attention"})
 
 
@@ -41,9 +41,9 @@ def _validate_transformers_compatibility() -> Version:
         or parsed not in _SUPPORTED_TRANSFORMERS_SPEC
     ):
         raise RuntimeError(
-            "RecurQuant's Qwen3.5 cache is tested only with stable releases in "
+            "RecurQuant's Qwen3.5 cache is tested only with "
             f"{_SUPPORTED_TRANSFORMERS_REQUIREMENT}; "
-            f"found transformers=={installed}. Install the tested range before creating "
+            f"found transformers=={installed}. Install the tested release before creating "
             "the cache."
         )
     return parsed
@@ -177,7 +177,7 @@ def create_qwen35_packed_cache(
     """Create a validated packed recurrent-state cache for Qwen3.5 inference.
 
     This factory covers text-only ``Qwen3_5ForCausalLM`` models on the tested stable
-    Transformers minor release. Passing a model additionally validates evaluation
+    Transformers release. Passing a model additionally validates evaluation
     mode, eager attention, and single-device materialization. Passing only a config
     validates its layer structure; it cannot validate the eventual model runtime.
     Every model forward must run under ``torch.inference_mode()`` or
@@ -213,3 +213,36 @@ def create_qwen35_packed_cache(
         layer_specs=layer_specs,
         record_evidence=record_evidence,
     )
+
+
+def create_qwen35_v02_mixed_cache(
+    model_or_config: Qwen35Source,
+    *,
+    record_evidence: bool = False,
+) -> PackedRecurrentStateCache:
+    """Create the frozen v0.2 mixed-precision Qwen3.5 cache.
+
+    The policy stores model layer 0 at INT8 and every other supported recurrent
+    layer at INT4, with group size 128, FP16 scales, nearest rounding, and seed
+    2339. It is the fixed development policy reported by the repository, not an
+    automatic selector for other checkpoints.
+    """
+
+    return create_qwen35_packed_cache(
+        model_or_config,
+        bits=4,
+        group_size=128,
+        scale_bits=16,
+        rounding="nearest",
+        seed=2339,
+        layer_specs={
+            0: QuantizationSpec(
+                bits=8,
+                group_size=128,
+                scale_bits=16,
+                rounding="nearest",
+                seed=2339,
+            )
+        },
+        record_evidence=record_evidence,
+    )