Skip to content

CLAUDE.md

Latest commit

 

History

History
30 lines (19 loc) · 13.9 KB

CLAUDE.md

File metadata and controls

30 lines (19 loc) · 13.9 KB
Raw

CLAUDE.md — shannon-prime-lattice

★ CURRENT STATE (2026-07-01). Nav: papers/START-HERE.md → papers/VERIFIED-SCOREBOARD.md → papers/PPT-LAT-KEYSTONE.md → papers/PPT-LAT-Roadmap.md → papers/PPT-LAT-FINDINGS-LEDGER.md → papers/PPT-LAT-ADR-002-DECIDE-EXECUTE-SPINE.md (governing law). FIVE repos: lattice (umbrella/docs/OKFS), shannon-prime-system (math core + L1 ABI), shannon-prime-system-engine (sp-daemon + CUDA + served chat + recall/agency), shannon-prime-harness (Python agent harness), Position_Is_Arithmetic (public papers). Read the foundation docs before trusting the "Current edge" notes below (session history).

This is Shannon-Prime. Read prompt.md in this repo FIRST — it is the canonical session bootstrap (what the project is, current state, methodology, the machine, the doc map, how the operator works). This file is the short version.

Repo role: umbrella / docs / OKFS — papers/ (the entire canonical doc set: STATE, KEYSTONE, Roadmap, RFC-001, the ADRs, the contracts, VERIFIED-SCOREBOARD, FINDINGS-LEDGER, START-HERE, the SWARM design), demos/, tests/, integration glue, and the two knowledge stores (SP-OKF + MEM-OKF). The math core is shannon-prime-system; the engine is shannon-prime-system-engine; the code lives in the other repos.

Governing law: ADR-002 — the Decide→Execute spine (papers/PPT-LAT-ADR-002-DECIDE-EXECUTE-SPINE.md): DECIDE in latent, EXECUTE in clean symbol/text, NEVER fuse latent content into generation, deciders don't execute.

Current edge (2026-07-01): FAITHFULNESS AXIS CLOSED END-TO-END + SWARM/DHT RE-ELEVATED PRIMARY. Faithfulness is now solved on BOTH general-knowledge conflict AND private zero-prior data. The recall selector graduated to L5-cosine — the fact signal is layer-localized in gemma4 global layer 5; a live cosine selector (SP_RECALL_L5, τ=0.30, default-off) obeys episodic memory over parametric priors at 86.89% paraphrase (G-L5-RECALL-LIVE, engine d9099cd). The heavy generative judge is PARKED — the hard-foreign kill-test showed 0 benefit over L5-direct+τ and it PASSed 15/18 (G-HARDFOREIGN-L5DIRECT/-JUDGE); K=2 was the minimum-working config but recall-thru-judge was the cost. The one hole native robustness does NOT cover — zero-prior/private data (the SNE crucible: 80% confabulation + 5% secret-leak on novel entities) — is closed by a deterministic attribute-grounding gate (SP_RECALL_ATTR_GATE τ=0.5) + a query-token guard (paraphrase-safe) + a ZERO-INFERENCE symbolic decline (G-SNE-ATTRGATE-ZEROINF, engine fc2e846: confab 80→0, leak 5→0, recall 100%, paraphrase untouched — the decline streams a fixed string with NO gemma4 forward, so hallucination is mathematically impossible). New doc papers/PPT-LAT-FINDINGS-LEDGER.md = the flat lookup table for every measured constant/layer/lever/boundary + per-test ledger (which layer, what τ/K, where the lever is — without re-deriving). SWARM/DHT re-elevated to a PRIMARY forward axis alongside ADR-002 (papers/PPT-LAT-DESIGN-SWARM-MEMORY-MESH.md): private, invite-only, content-addressed, signed replication of the MEM-OKF store across the operator's nodes (rides byte-exact addressing + the receipt ledger; audited crypto — libp2p noise / X25519 / Ed25519 / ChaCha20-Poly1305, NOT rolled field-crypto). Verified state: papers/VERIFIED-SCOREBOARD.md (10 VERIFIED · 0 PARTIAL · 0 false-greens; NIGHTSHIFT graduated PARTIAL→VERIFIED 2026-07-02, receipts re-checked 2026-07-02). Canonical run config: papers/RUNBOOK-ONE-CONFIG.md — GREEN-LIVE 2026-07-02 (G-ONECONFIG-LIVE: the whole composition gated in one run; systemecho delivery = 88.52% obey / 0 leak, delivery solved, obey ceiling = the selector); latest audit: papers/AUDIT-2026-07-02.md.

Prior edge (2026-06-21): NIGHTSHIFT OFFLINE CURATOR GREEN + MEM-OKF anti-rebuild store ACTIVE. run_kairos_curator (engine 6107f3e, default-off SP_NIGHTSHIFT_OFFLINE): the 12B model-call ep.secret extractor → teacher-forced causal-ablation admit (TAU=−8) → conformant MEM-OKF emit. G-NIGHTSHIFT-CURATOR GREEN on the synthetic gate (novel "8-FALCON-7729" collapse −33.59 ACCEPT / parametric "Paris" 0.00 REJECT, ~33-nat sep; emit rc=0, addr-join verified) — criteria 1-4; criterion 5 (live B4 in-distribution) PENDING (gated-GREEN/default-off, NOT yet running on real chat turns). The model-call extractor (not token-rarity) gives the surgical secret so the ablation measures fact-dependency, not context-destruction. The causal ablation oracle (TAU=−8) is the official admission gate; the learned latent W_c head (SP_B3_WC) is the live recall selector; the native Diffusion Judge is in the drawer pending the OOD kill-test. MEM-OKF (tools/okf_mem.py + memory-okf/, spec papers/MEMORY-OKF-PROFILE.md) = the content-addressed LUT→summary→full anti-rebuild store; the lookup-before-you-build pre-flight is in prompt.md §0/§8 + the non-negotiable below. Build steps: engine 9ad7ede→9ee4668→6107f3e; record papers/CONTRACT-NIGHTSHIFT-CURATOR.md + STATUS-MAP-2026-06-21.md.

Prior edge (2026-06-20): B3-WC AUTONOMOUS-RECALL CAMPAIGN RESOLVED — a learned W_c head does LIVE, instance-level episodic recall on the served Gemma-4-12B chat, with clean foreign-reject (engine edc8079, pushed; receipts tests/fixtures/chat_fullstack/G-CHAT-B3-WC-{DEPLOY,DIV2}.log; record papers/CONTRACT-CHAT-FULLSTACK.md + SESSION-HANDOFF.md §0d). The served chat (CONTRACT-CHAT-FULLSTACK — coherent + byte-exact + O(1) + single-entry, GREEN 2026-06-19) now has an autonomous librarian. The arc is a boundary-thesis win (negatives attached): ALL hand-designed relevance signals FAIL open-world (6 verifiers + 4 Disposer signals + cosine-qK + ΔLL-polarity, dominated by per-episode bias at N=3 wiki, 4dba6c8/2b623ab/acd7b3a); ROOT CAUSE = wiki facts are PARAMETRIC → episodic dependency unmeasurable → the corpus was the wall (B3-v10 27038d3); THE UNLOCK = teacher-forced ablation knockout on a NOVEL needle (SP_B3_SECRET cudaMemset-ablates the secret's source KV rows, ΣΔLL with/without → novel −33.56 vs parametric −0.15, B3-v13 3-archetype matrix TAU −8.0, 15738c1/b6470cc) which is both a SHIPPABLE gate and a PERFECT LABELER (7556d04). Labels → curator-minted corpus (split-entropy minter, secret stated ONCE, 200 needles + 60 foreign, 52ec468/adfcb06/f4166c7) → learned W_c (HD=512→r=32; logsumexp-over-positions then mean-over-heads relevance; InfoNCE over [episodes+NULL/s0] + reject hinge). Corpus DIVERSITY was the binding constraint at every granularity: unique-subject needles took instance top-1 34%→100% (mint_corpus_v2 72d9c77/f62e6ef); the int16 "degradation" was a wrong-reduction bug — logsumexp-mean=361/361, int16==f32. G-CHAT-B3-WC-DIV2 (87044d8): 360/361 recall + 50/50 foreign reject, int16-exact, s0=+0.102. G-CHAT-B3-WC-DEPLOY (edc8079, LIVE): recall.rs WcHead/wc_score + routes.rs SP_B3_WC ((E+1)-NULL-argmax → replay winner @SP_REPLAY_MTARGET=42 or clean prompt; default-off = null floor); matched → RECALL ep_n_div_000 (9.858), foreign → NULL → clean "Paris." run_console_recall.bat launcher + console stop-button. NEXT = B4 NIGHTSHIFT (between-turn consolidation, DEFERRED, pre-scoped SESSION-HANDOFF §0d). Prior edge (2026-06-18): TWO closes the same day — (A) XBAR memory UNIFIED onto the exact-integer O_K substrate, and (B) the BYTE-EXACT FORWARD CLOSED GREEN on the real gemma-4-12B. (B) the gemma-4-12B forward now runs exact-integer end-to-end: the four nonlinear islands (RMSNorm/softmax/GELU/RoPE) + attention convert to exact-integer on the SAME dual-prime CRT-NTT as the memory ring (q1=1073738753 q2=1073732609, M≈2⁶⁰ fits u64 ⇒ no __int128), behind default-off SP_BYTEEXACT (one-shot decode byte-untouched = null floor). byte-exact = EXACT ARITHMETIC / cross-machine determinism / AUDITABILITY, NOT compression. G-BYTEEXACT-FORWARD-12B GREEN (OFF 4.6665 byte-identical null floor / ON 4.6569 parity / run-to-run bit-identical); daemon drives the 12B prefill + decode via the NEW L1 verb sp_session_register_kvdecode_backend (G-WIRE-CUDA-DECODE-GEMMA4 32/32 == oracle). The byte-exact linear algebra was ALREADY in crate tools/sp_dsp_smoke (don't re-derive); new piece = the 4 islands (sp_islands_q_ref.rs + core/exact_islands/). Only the external 2-physical-GPU check remains. Engine 69c0588 + submodule d9d96f3; record papers/CONTRACT-BYTEEXACT-forward.md + SESSION-HANDOFF.md §0b. (A) This session re-carried the whole memory stack off generic float carriers onto the engine-native dual-prime negacyclic CRT-NTT (core/ntt_crt + core/poly_ring; frozen primes q1=1073738753 q2=1073732609 M=1152908312643096577). G-R3-BIND-on-O_K (engine 0019b86): Ring-3 VSA bind 256/256 bit-identical to native sp_pr_mul/ntt/sp_pr_score_kstore, ±1 carrier int==float, reduction-order-immune (M byte-identical across permutations vs float 4.44e-15 drift). G-R3-ORGANISM-NATIVE (1f0f6be): dualroute + nightshift now run on native sp_pr_mul (D=1024 = two 512-blocks; CAP=32). G-R2-FROB (dbe4103/d076797): Frobenius π^k INTEGER Ring-2 episode store, rank-2 O_K lattice (a16 ~lossless / a16b8 sub-ULP relL2 1.2e-7, 0.76× store) — "lossless" by reconstruction fidelity (the n=42 PPL gate is blind below ~1%; no fake +0.000%). G-XBAR-ORGANISM-FULL (15e7051): full loop on REAL episodes — continuous audio (EAR) → C2 256-bit sig → native integer Ring-3 superposition (+text decoys) → audio-cue top-1 retrieve → C2 Hamming verify (accept audio / reject text) → Frobenius integer store → continuous float lands clean into the 12B resident cache (checks=5 fails=0). G-PERIOD6-REBASE (d2d7ceb): C2/Ring-3 content-hash period 8→6 to the true gemma4 global layers {5,11,…,47}, re-gated GREEN. BOUNDARY THESIS: O_K wins on EXACT ARITHMETIC (the container); every structure-on-content lever is measured-inert and kept as an honest negative — Leg B split-prime Dirichlet carriers (d7d96fe, operationally inert), Möbius-on-M (1e70763, sheds memories 1.000→0.969@N=32), entropy-coding the Frob codes (e6d17bb, 1.02× dead weight), T2-Möbius on the real 12B embedding (ac76c8e, recon cos 0.032 == random). Tools tools/ring3/{ok_bind.py,g_r3_bind_ok.py,g_xbar_organism_full.py} + tools/curator/frob_episode.py; receipts tests/fixtures/xbar_r3/ + tests/fixtures/xbar_organism/. NEXT (prior framing): T4 Frobenius π^k of the model WEIGHTS (since CONVICTED redundant vs OK_Q4B — see PPT-LAT-STATE) — the live NEXT is B4 NIGHTSHIFT, then KAIROS post-organism. Live state: SESSION-HANDOFF.md + CURRENT-STATE-OF-PROJECT.md.

Non-negotiables (full detail in prompt.md §3/§4/§7):

  • Receipts-first. No number without a reproducing command + a LEDGER.md row. Bit-exact-when-off. Scope travels with every figure. Gates: parity / deflection(<2%) / poison; telemetry-then-pin; no silent gate revision — surface upstream; honest negatives stay attached.
  • Check the code + commits + git status/git fetch BEFORE trusting memory or a summary. STATE (papers/PPT-LAT-STATE.md) is the PROVEN record — read it, trust it, re-prove only with concrete cause.
  • Gemini is a valued collaborator but we verify — read the actual paper/code, fix/improve its suggestions, map onto our discrete substrate; never blindly adopt.
  • Drive by default. Make the obvious call; surface only genuine forks, and recommend. No filler, no closers.
  • Anti-contamination: do not copy from shannon-prime/ or shannon-prime-engine/.
  • SP-OKF knowledge discipline. All knowledge docs are SP-OKF concepts (our profile of Google's Open Knowledge Format v0.1): carry the frontmatter (type required + title/description/tags/timestamp/resource + the receipts-first sp_status/sp_gate/sp_commit/sp_repro). New types register in papers/SP-OKF-PROFILE.md §2 FIRST. Run the validator (gate G-OKF-CONFORM) on a touched bundle before commit: python tools/okf_validate.py <bundle-dir>. Spec + phased rollout: papers/SP-OKF-PROFILE.md.
  • MEM-OKF anti-rebuild PRE-FLIGHT (binding). Before building ANY subsystem: python tools/okf_mem.py lookup --root memory-okf <keyword> + grep the tree. A new file for a capability that already exists is a defect — this project has rebuilt the same subsystems 20+ times. Content-addressed LUT→summary→full store (tools/okf_mem.py + memory-okf/); spec papers/MEMORY-OKF-PROFILE.md; gate python tools/okf_mem.py verify --root memory-okf. At session end, bank durable "X already exists — don't rebuild" facts via okf_mem.py add.

Start: read prompt.md → papers/PPT-LAT-STATE.md + papers/PPT-LAT-Roadmap.md + active contract → MEMORY.md → check the tree → confirm next falsifiable step → execute, gate, commit/push, bank memory.

Build from clean (engine): CPU/math-core builds with clang-cl (MSVC-ABI, supports __int128 in exact_islands.c), NOT cl.exe (which can't — C4235). env-cpu.bat is pinned to clang-cl; do not revert it (that drift = clean-build RED, a recurring cause of restarts). Canonical from-clean chain + every gotcha + gate G-CLEAN-BUILD: papers/BUILD-ENV-TOOLCHAIN.md (verified GREEN 2026-06-28).

Environment & credentials (added 2026-06-11): the toolbox — compute lanes (RunPod=bake / Colab=prototype / gws=Google APIs), the three shells and their traps, storage law, account map — lives in ENVIRONMENT.md (this repo, read it before touching any cloud). Secrets live ONLY in D:\F\shannon-prime-repos\archive\notes_and_stuff\creds\claude-credentials.txt (outside all repos; paths-not-values everywhere else; renewal procedures inside). Where things stand right now — in-flight runs, the decision queue — is SESSION-HANDOFF.md (this repo; update it at every session end).