deploy: implement int8 predictor and int4 Carbon quantization

[AbdelStark]

Context

Hits the on-device memory / latency budget.

Scope

• Per-tensor symmetric int8 for predictor + action encoder with calibration over 1k reference windows.
• Q4_K_M for Carbon via llama.cpp toolchain.
• Evaluate quality drop against RFC-0016 §3.3 budget.

Out of Scope

• AWQ / GPTQ alternatives (Phase 4 if needed).

Design Reference

• RFC: rfcs/0010-on-device-personal-genome-deployment.md §3.3
• RFC: rfcs/0016-performance-budget.md §3.3

Acceptance Criteria

• Int8 predictor: ≤ 1.0 AUROC drop on ClinVar coding
• Int4 Carbon + int8 predictor: ≤ 2.0 AUROC drop on ClinVar coding
• Quantization is reproducible from a seeded calibration set

Parent tracking issue: #14

[AbdelStark]

Blocked (session-scoped): requires runtime toolchains not present in this environment.

This issue depends on ONNX / Core ML / GGUF / quantization tooling whose installation, version pinning, and host-OS support is non-trivial:

• ONNX Runtime + onnx-converter for the export path.
• Core ML Tools (macOS-only conversion path).
• llama.cpp's convert.py family for GGUF.
• bitsandbytes / optimum-quanto for int8/int4 calibration.

Beyond installation, the issue requires a trained predictor (predictor.safetensors) to convert — which itself depends on #41 (predictor) and #44 (trainer) landing.

Foundations now in place that the export pipeline will plug into:

• geno_lewm.errors.ExportFormatError / QuantizationError / BackendUnsupportedError for typed failures.
• geno_lewm.provenance.{sha256_file,Manifest} for the post-export model_id derivation that the runtime loader (#66) will compare against.

To unblock: install the target toolchain(s), wait for #41 / #44 to ship a checkpoint, and pick this up.

[AbdelStark]

Blocked in the current repo state.

Quantization cannot be implemented and accepted production-grade until the checkpoint, export, and evaluation prerequisites exist:

• The issue requires per-tensor int8 calibration over 1k reference windows and Q4_K_M Carbon conversion. There is no trainer-produced checkpoint artifact yet; training: complete clean-machine Carbon-backed trainer run #44 remains open and labeled status:blocked.
• The deploy artifacts it quantizes/export-wraps depend on the per-target export path; deploy: implement ONNX export #67/deploy: implement Core ML export #68/deploy: implement GGUF export for CPU-only runners #69 are blocked pending the checkpoint/toolchain contracts.
• Acceptance requires AUROC-drop measurement on ClinVar coding. The ClinVar/eval path is still blocked by the open eval/data issues (eval: run real ClinVar coding/non-coding benchmark split #53/eval: add hosted smoke-regression gate #54 and related data loaders), so there is no implemented quality gate to compare quantized vs bf16 checkpoints.
• Current dev toolchain probe: torch: False, onnxruntime: False, bitsandbytes: False, optimum: False, optimum.quanto: False, safetensors: False (pyarrow: True).

Unblock once checkpoint production/loading, export targets, calibration fixtures, and the ClinVar AUROC evaluation gate exist with pinned quantization tooling. Then implement seeded quantization and validate the ≤1.0 / ≤2.0 AUROC-drop budgets from RFC-0016.