Gemma v4 physical qualification: execute the frozen prediction-aligned campaign

[ezutfen]

Objective

Execute the first physical Gemma v4 qualification campaign under the methodology frozen by issue #95 / PR #96.

Accepted starting point:

• issue S5 doctrine: define acceptance-bearing numerical core vs diagnostic tensor telemetry #93: NUMERICAL_CORE_TWO_TIER_DOCTRINE_ACCEPTED;
• issue Gemma v4 methodology: freeze prediction-aligned two-tier qualification before physical execution #95: GEMMA_V4_PREDICTION_ALIGNED_METHODOLOGY_FROZEN;
• accepted Gemma v4 methodology: freeze prediction-aligned two-tier qualification before physical execution #95 implementation head: e056b2ae46f4c36da8ae93a3beb0b38c820a283e;
• PR feat: freeze Gemma v4 prediction-aligned methodology #96 merge: e12a6e3d5589044bace0c9555c0d364fb57a6229;
• historical issue Gemma v3 physical qualification: execute frozen decision-stability methodology #88 remains immutable V3_HOLDOUT_FAIL;
• consumed h86-* evidence remains diagnostic-only and permanently ineligible for this campaign.

This issue authorizes physical reference/candidate calibration execution and, only after a later explicit maintainer authorization in this issue, the single-use h95 holdout unseal and holdout execution.

Do not change the frozen methodology.

Pre-execution synchronization

Before any CUDA/model execution:

1. sync ROADMAP.md from the stale pending maintainer acceptance wording to accepted Gemma v4 methodology: freeze prediction-aligned two-tier qualification before physical execution #95 state;
2. record PR feat: freeze Gemma v4 prediction-aligned methodology #96 merge e12a6e3d5589044bace0c9555c0d364fb57a6229 and terminal methodology disposition;
3. verify every committed v4 methodology/schema/manifest hash from main;
4. verify the h95 ciphertext, certificate, custody-record bytes, and external custody remain exactly the frozen identities;
5. perform an applicability census of the physical reference/candidate environments against the frozen subject. Any material correctness-relevant dependency drift stops the campaign for maintainer adjudication rather than silently changing the subject.

No physical result may be used to modify the methodology, sample size, tier classification, threshold rule, decision-domain construction, semantic theorem, or holdout.

Frozen physical subject

Retain the issue #95 subject unless pre-execution applicability validation stops the campaign:

• model: google/gemma-4-12B-it;
• revision: 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7;
• checkpoint SHA-256: 5a84cb313260ac447237b890387116dfa8682e49a6b44bc585ae8353abbff18d;
• native BF16 text execution;
• Triton attention;
• deterministic greedy replay-prefill;
• exactly 8 canonical-prefix decisions/case;
• replay chunk within the frozen <=64 row regime;
• reference: accepted RTX 3090 reference node/path;
• candidate: accepted three-stage RTX 3060 chain/path.

Capture exact node/device/software/backend/topology identity before execution and bind it into evidence.

Frozen v4 statistical / comparator contract

Statistical calibration

• c95: exactly 1,896 cases;
• exactly 24 frozen population cells;
• exactly 79 independent cases/cell;
• independent unit = case;
• selected stress contributes zero predictive sample size.

Stress

• p95: exactly 48 reference-only cases, 2/cell;
• compute the frozen reference margin only;
• reject negative/nonfinite margins according to the frozen selector contract;
• select exactly four smallest finite nonnegative margins + four largest;
• deterministic case-ID tie break;
• freeze/commit/push/refetch the selected-eight identity before candidate stress execution.

Acceptance-bearing core

Exactly four scalar families:

1. fp32-consumer-logits:max-absolute-difference;
2. fp32-consumer-logits:rms-difference;
3. fp32-consumer-logits:p99-absolute-error;
4. decision_local_E_D / per-case max over all 8 canonical-prefix decisions.

Mandatory telemetry

Exactly the twelve internal family/metric identities classified by accepted #93. They remain mandatory retained evidence, but a finite reference-band exceedance alone is not a qualification failure.

NaN/Inf at any finite_required identity remains fail-closed.

Phase A — physical calibration

A1. Reference execution

Run the frozen reference path for:

• all 1,896 c95 statistical cases;
• all 48 p95 stress-pool cases.

For each applicable decision/case retain the evidence required by #95, including:

• exact model/case/plan/state/path/position attribution;
• canonical reference prefixes;
• full-vocabulary FP32 consumer logits;
• all 15 frozen family/metric observations for core + telemetry accounting;
• reference decision-domain memberships/hashes;
• frozen argmax/tie semantics;
• reference margins for p95 selection;
• finite/integrity/path evidence.

No candidate observation may influence stress selection or decision-domain membership.

A2. Freeze selected stress

Mechanically apply the frozen selector to p95 reference evidence.

Commit, push, and byte-refetch:

• reference-margin summary;
• selected-eight manifest;
• selection/provenance hashes.

Only the selected eight receive candidate stress execution.

A3. Candidate calibration execution

Run the frozen heterogeneous candidate for:

• all 1,896 c95 cases;
• exactly the selected 8 p95 stress cases.

Candidate execution must be teacher-forced on the exact reference canonical prefix at all 8 decisions.

Retain:

• exact integrity/attribution/path evidence;
• full-vocabulary candidate FP32 consumer rows;
• actual full-vocabulary emitted/winner identity under frozen argmax semantics;
• all 15 numerical family/metric observations;
• decision-local errors on frozen reference domains;
• complete mandatory telemetry;
• zero silent fallback/substitution/path drift.

Calibration adjudication

After complete valid calibration evidence, use only the frozen issue #95 evidence-consuming deriver.

It must independently:

• validate 24×79 c95 balance and exact identities;
• validate exact selected-eight provenance;
• validate all 8 decision rows per case;
• recompute every case_E_D;
• derive the statistical/stress maxima itself;
• emit exactly four core limits;
• emit exactly twelve telemetry reference bands;
• bind exact evidence, comparator, holdout commitment, custody, decision-domain, and derivation-program identities.

Core and telemetry reducers remain prospectively frozen as max(statistical_max, selected_stress_max).

Calibration must stop before holdout unless all of the following pass

• exact applicability/integrity/path prerequisites;
• all 1,896 statistical cases complete;
• exactly 8 selected stress cases complete;
• all required values finite;
• all mandatory telemetry present;
• all canonical-prefix decision rows complete;
• zero DECISION_DOMAIN_ESCAPE;
• zero semantic stable/ambiguity failures;
• E_D derived without manual edit/rounding;
• core-threshold and telemetry-band artifacts pass their frozen schemas;
• threshold/band artifacts are committed, pushed, and byte-refetched;
• h95 remains SEALED_NOT_CONSUMED.

A valid calibration failure is terminal for this campaign. Do not tune and retry.

Mandatory holdout stop / maintainer gate

After threshold/band freeze and successful non-decrypting h95 preflight, STOP.

Do not decrypt h95 yet.

Post a maintainer-review checkpoint containing at minimum:

• exact physical producer SHA(s);
• environment/applicability record;
• reference completion counts;
• selected stress identity;
• candidate calibration completion counts;
• semantic calibration counts;
• exact four core limits;
• telemetry-band manifest identity;
• threshold/band artifact SHAs;
• h95 ciphertext/certificate/custody identities;
• preflight verdict PRECONDITIONS_PASS_STOP_BEFORE_DECRYPT or precise failure;
• explicit statement that no holdout plaintext has been revealed.

Use a narrow ready state such as:

V4_HOLDOUT_UNSEAL_READY

Only an explicit maintainer authorization after reviewing this checkpoint may continue to Phase B.

Phase B — single-use h95 holdout (not authorized until maintainer says so)

After explicit maintainer authorization only:

1. perform the one-time h95 decrypt from the already-verified external key/custody path;
2. verify plaintext/secret/case identities against the frozen public commitment;
3. run the reference path for exactly the 24 h95 cases, one/cell;
4. run the candidate teacher-forced on the exact reference canonical prefixes;
5. evaluate using only the already-frozen core limits, telemetry bands, and semantic contract;
6. do not modify any threshold or methodology after observing holdout values.

Holdout acceptance

A successful campaign requires every one of the 24 holdout cases to satisfy:

• exact integrity/applicability/path prerequisites;
• all finite-required identities finite;
• all four core limits with inclusive observed <= limit;
• zero DECISION_LOCAL_BOUND_EXCEEDED;
• zero DECISION_DOMAIN_ESCAPE;
• stable decisions require exact reference winner;
• unstable decisions require ambiguity-set membership;
• all twelve mandatory telemetry identities retained.

Telemetry reference-band exceedances are recorded as TELEMETRY_ALERT but do not independently fail qualification.

Any valid acceptance-bearing holdout failure is terminal. No post-result tuning, resealing, replacement holdout, or valid-failure rerun is permitted.

Invalid-run policy

A rerun is allowed only for evidence-supported invalid execution (for example interrupted transport, node failure, corrupt/incomplete artifact production, or demonstrable applicability violation before a valid result exists).

If an execution produced a valid correctness-bearing observation, an unfavorable result is not an invalid run and cannot be rerun away.

Preserve invalid-run evidence and reason codes rather than deleting inconvenient attempts.

Final evidence / verdict

Retain a machine-readable and human terminal report with exact producer/provenance hashes and complete core/telemetry/semantic accounting.

Use a narrow terminal result, for example:

• V4_QUALIFICATION_PASS;
• V4_CALIBRATION_INTEGRITY_FAIL;
• V4_CALIBRATION_SEMANTIC_FAIL;
• V4_CALIBRATION_APPLICABILITY_BLOCKED;
• V4_HOLDOUT_CORE_FAIL;
• V4_HOLDOUT_SEMANTIC_FAIL;
• V4_HOLDOUT_INTEGRITY_FAIL;
• V4_HOLDOUT_APPLICABILITY_FAIL.

Do not reuse #88's verdict identity and do not reinterpret #88.

Deliverables

Before the holdout-ready checkpoint:

• ROADMAP accepted-Gemma v4 methodology: freeze prediction-aligned two-tier qualification before physical execution #95 synchronization;
• physical environment/applicability record;
• reference c95/p95 evidence;
• frozen reference-margin + selected-eight stress artifacts;
• candidate c95/selected-p95 calibration evidence;
• decision-domain manifest;
• calibration summary;
• four-core threshold manifest;
• twelve-telemetry reference-band manifest;
• non-decrypting h95 preflight record;
• tests/validators needed to reproduce the retained adjudication.

After authorized holdout execution:

• immutable h95 unseal/execution evidence;
• core exceedance accounting;
• telemetry-alert accounting;
• semantic decision accounting;
• terminal report/verdict;
• ROADMAP update preserving historical Gemma v3 physical qualification: execute frozen decision-stability methodology #88 and the v4 result.

Non-negotiable constraints

• No methodology/threshold/sample/tier changes after physical observations.
• No c74/c86/past-holdout reuse.
• Selected stress never counts toward predictive sample size.
• No telemetry alert becomes an acceptance gate retroactively.
• No semantic pass waives an acceptance-bearing core exceedance.
• No core pass waives semantic/integrity failure.
• No hidden holdout access before explicit maintainer authorization.
• Historical R6 — falsify the InferSwarm architecture against dense Gemma 4 12B #65/Execute Gemma heterogeneous numerical qualification: calibration → threshold freeze → sealed holdout #76/Execute Gemma numerical qualification v2: reference selection → calibration → threshold freeze → retained holdout #81/Gemma v3 physical qualification: execute frozen decision-stability methodology #88/Post-v3 diagnosis: characterize the numerical-envelope holdout failure before successor qualification #90/S5 doctrine: define acceptance-bearing numerical core vs diagnostic tensor telemetry #93/Gemma v4 methodology: freeze prediction-aligned two-tier qualification before physical execution #95 remain immutable.
• Leave public planner/strategy APIs unfrozen.

Exit criteria

This issue is complete only after either:

1. a valid terminal calibration failure/block is retained; or
2. calibration passes, maintainer explicitly authorizes h95 unseal, and the single-use holdout produces a retained terminal v4 qualification verdict.

Do not merge an evidence PR before maintainer adjudication of the applicable terminal state.

[ezutfen]

V4_HOLDOUT_UNSEAL_READY — Phase A complete (maintainer gate)

Producer / PRs

• FreeToken producer SHA: 57dfcb7289efac8f66de5b3abbe8de04f2580f75 (unchanged; PR Issue #97 producers: Phase A calibration (57dfcb7) + Phase B holdout-only authority (c60a4b7) — terminal evidence retained FreeToken#31 OPEN, unmerged)
• InferSwarm PR Issue #97 v4 campaign: terminal V4_HOLDOUT_CORE_FAIL (Phase A pass, Phase B holdout executed) #98 head: e1d3a16 (branch issue-97-v4-physical-qualification), OPEN/unmerged
• CI: green through 470e675 (all subsequent commits are docs/artifact-only); CI-equivalent suites run locally on the freeze tree: YAML 4/4 valid, phase0 manifest OK, unittest suites 74/76/79/83/86 all OK, v4 methodology+thresholds 25 passed

A1 reference (inferswarm04 3090 + batched continuation)

• c95 statistical: 1896/1896 committed and content-verified (row SHA vs summary per case)
• p95 stress pool: 48/48 committed, run index verified, 0 NaN/Inf
• Invalid attempts retained: 1 SIGTERM-interrupted case (preserved), 1 stale index (preserved), 1 batch5 full-corpus duplicate run (817 duplicate realizations quarantined, excluded)

A2 freeze (committed + byte-refetched)

• reference-margin-summary sha 09bf9299dcb605a7ddab11cafec0d0583a288f55733e3a3f023a8021bd167b15782c
• selected-eight sha a1bb5caadbf3bab0dd35f34db5fa44d6e0a1d3bed67648b32b4b97969d3f44d2
• decision-domain manifest sha 59a7b740e5724cc5d69206d2dfbb7ae3c87013f5292c3e07db92081f965383e9 (1896 + 8 rows × 8 decisions)

A3 candidate (teacher-forced 3-stage 3060 chain)

• 1896/1896 c95 + 8/8 selected p95 committed; teacher-forcing assertion passed on every decision

Calibration + thresholds (frozen deriver bc3dc5ecf9f1d6f15a9e4c28ae6701535cc35da43af1a7701b40f853f8ac039b)

• calibration-summary sha c5d81c188018eabaa1cafec0d0583a288f55733e3a3f023a8021bd167b15782c
• Four core limits (max(statistical_max, stress_max), inclusive):
  • fp32-consumer-logits:max-absolute-difference = 26.625 (0x1.aap+4) — statistical-binding
  • fp32-consumer-logits:rms-difference = 10.1346… (0x1.444eb652ef62bp+3)
  • fp32-consumer-logits:p99-absolute-error = 16.640625 (0x1.0ap+4)
  • decision_local_E_D = 26.625 (0x1.aap+4)
• core-threshold-manifest sha c714f5613b541106e985c7337e723dcad297637541e5f937bb2c6f3a7bfbbf50
• telemetry-reference-bands sha 7b38c13e11416f81139c1721f433a12d4e4377540bb115b63e09f0bb54032534 (12 bands)

Semantic adjudication (15,232 decisions)

• stable 3,772 / unstable 11,460
• stable exact-winner failures: 0; ambiguity-membership failures: 0
• DECISION_LOCAL_BOUND_EXCEEDED: 0; DECISION_DOMAIN_ESCAPE: 0

h95 state

• ciphertext 17757acb091fd5b3b1a4ef7b8fcd392b448e91eee39c04003afd9a4ccac4a844, certificate 99bb4eb992d3a38eb54f6821c4477edc7ad30df977333d39a5b120dac8e57430, pubkey DER 629b2ab2320cfe1517b5eb435edf241cf30d109b8da0bc9b62002165d6a9fdd2 — unchanged
• Non-decrypting preflight: PRECONDITIONS_PASS_STOP_BEFORE_DECRYPT
• h95 remains SEALED_NOT_CONSUMED; unseal authorization remains false; no h95 plaintext revealed; no h95 case executed

STOP. Awaiting explicit maintainer authorization for Phase B (h95 unseal + holdout execution).

[ezutfen]

V4_HOLDOUT_EXECUTION_AUTHORITY_READY_STOP_BEFORE_DECRYPT — Phase B authority closed (maintainer gate)

Follows the maintainer classification of the 2026-09-06 stop as V4_HOLDOUT_UNSEAL_READY_BLOCKED_BY_MISSING_EXECUTION_AUTHORITY. The gap (no h95 execution path in the frozen producer) is now closed with a separate holdout-only producer. h95 was not decrypted.

Producers (FreeToken PR #31, OPEN/unmerged)

• Phase A calibration producer (immutable): 57dfcb7289efac8f66de5b3abbe8de04f2580f75 — unchanged; all four frozen benchmarks/inferswarm_97/ files byte-identical to their freeze sha256 (mechanically asserted by test)
• Phase B holdout-only producer (new): c60a4b7080913eba90585de8e8602460a7f5b1a7 (parent = Phase A head)
  • adds benchmarks/inferswarm_97b/ h95-only entrypoints: accept exactly the 24 committed h95 cells, reject all c95-/p95-/historical/unknown/substituted/duplicate IDs before any CUDA work
  • delegates to the frozen Phase A runner main() with ONLY the admission callable swapped — zero execution/model math added, no methodology/threshold/topology/semantic/evidence-contract change (source-audited by test)

Equivalence evidence (CPU-only, synthetic fixtures, no secret material)

• 13-test suite tests/research/test_inferswarm_97b_holdout_authority.py: Phase A byte-identity, holdout admission matrix, delegation/exit-code propagation, Phase A package unaffected, no-torch/no-execution-math source audit — all green on the c60a4b7 tree
• Phase A harness suite (10 tests) still green; InferSwarm v4 methodology+thresholds suites (25 tests) still green at the new evidence head
• Phase B file sha256s verified against the remote raw bytes at c60a4b7 (all four match)

Records (InferSwarm PR #98, OPEN/unmerged, head 7706480)

• docs/qualification/gemma4-12b-it-v4-campaign-97/PHASE-B-EXECUTION-AUTHORITY.json — binds both producers, all unchanged frozen identities (ciphertext 17757acb…, certificate 99bb4eb9…, custody 0c9746e7…, pubkey DER 629b2ab2…, core thresholds c714f561…, telemetry bands 7b38c13e…, decision-domain 59a7b740…, selected-eight a1bb5caa…, calibration summary c5d81c18…, deriver bc3dc5ec…), byte-identical Phase A file hashes, topology, and the unseal rule
• Non-decrypting preflight re-run 2026-09-06: PRECONDITIONS_PASS_STOP_BEFORE_DECRYPT (all five identities reproduce)

h95 state

• SEALED_NOT_CONSUMED; unseal authorization remains false; no plaintext revealed; no h95 case executed; no decrypt performed

STOP. Awaiting a NEW explicit maintainer authorization to decrypt h95 and execute the frozen 24-case holdout (reference 3090 + teacher-forced 3060 chain) under the Phase B producer c60a4b7….

[ezutfen]

V4_HOLDOUT_CORE_FAIL — Phase B terminal verdict (issue #97 complete)

Executed under the explicit Phase B authorization against authority head 7706480, Phase A producer 57dfcb72…, Phase B producer c60a4b70…. All frozen identities, clean node worktrees at c60a4b7, and PRECONDITIONS_PASS_STOP_BEFORE_DECRYPT were re-verified immediately before decrypt.

h95 unseal + commitment verification

• Single-use decrypt performed 2026-09-06 from the verified external custodian path (CMS DER, RSA recipient). h95 is now CONSUMED.
• Commitment verification before any model execution: ALL PASS — 24 cases, every per-case prompt/token_ids/case sha256 recomputed against the frozen public commitment; seed/generator/tokenizer identities bound. Plaintext never entered either repository.

Execution (zero invalid attempts)

• Reference (inferswarm04 3090): 24/24, 0 NaN/Inf, all 192 decision rows sha-verified
• Candidate (3-stage 3060 chain): 24/24, 0 NaN/Inf, teacher-forcing prefix identity proven on all 192 decisions, forced trajectories byte-equal to reference

Core adjudication (frozen limits, inclusive observed <= limit)

family	observed	limit	verdict
fp32-consumer-logits:max-absolute-difference	23.296875	26.625	PASS
fp32-consumer-logits:rms-difference	8.920100	10.134608	PASS
fp32-consumer-logits:p99-absolute-error	16.765625	16.640625	FAIL (+0.75%)
decision_local_E_D	20.875	26.625	PASS

Semantic: 192/192 SEMANTIC_PASS (all unstable-ambiguous, admissible); 0 STABLE_DECISION_MISMATCH, 0 UNSTABLE_DECISION_INADMISSIBLE, 0 DECISION_LOCAL_BOUND_EXCEEDED, 0 DECISION_DOMAIN_ESCAPE.

Telemetry: 4 TELEMETRY_ALERT (non-qualification-failing), all on h95-01-01-01: hidden-residual-stream:rms-difference, final-normalized-hidden-state:rms-difference and :p99-absolute-error, bf16-logits:p99-absolute-error.

Failure anatomy: one case — h95-01-01-01 (ordinary-prose, shortest 4-8-token regime) — drives the p99 exceedance, the observed max-abs/E_D maxima, and all four telemetry alerts. Every other case sits ≥3x below the failing limit (next p99: 5.25). Consistent with the #88/#90 heavy-tail finding; #88 remains immutable and un-reinterpreted.

Terminal per doctrine: valid acceptance-bearing failure; no rerun, no tuning, no reseal, no replacement holdout.

Retained evidence (InferSwarm PR #98 head de7c6012, OPEN/unmerged):

• docs/qualification/gemma4-12b-it-v4-campaign-97/b/TERMINAL-REPORT.md
• docs/qualification/gemma4-12b-it-v4-campaign-97/b/h95-unseal-record.json
• docs/qualification/gemma4-12b-it-v4-campaign-97/b/holdout-adjudication.json
• docs/qualification/gemma4-12b-it-v4-campaign-97/b/holdout-rows.json (24 cases × 15 envelopes + 8 decision rows each)
• Node-local full evidence: 04 /srv/inferswarm/state/issue97/holdout-reference/ (2.0G), 01 /srv/models/issue97/holdout-candidate/ (416M) + holdout-adjudication/, 03 /srv/models/issue97/holdout-candidate-laststage/ (3.6G, candidate FP32 rows)

FreeToken PR #31 (Phase A+B producers) and InferSwarm PR #98 remain OPEN and unmerged for maintainer review.